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Abstract 

We briefly review the formal picture in which a Calabi-Yau n-fold is the com- 
plex analogue of an oriented real rt-manifold, and a Fano with a fixed smooth 
anticanonical divisor is the analogue of a manifold with boundary, motivating 
a holomorphic Casson invariant counting bundles on a Calabi-Yau 3-fold. We 
develop the deformation theory necessary to obtain the virtual moduli cycles of 



|LT|, |BF] in moduli spaces of stable sheaves whose higher obstruction groups 
vanish. This gives, for instance, virtual moduli cycles in Hilbert schemes of 
curves in P^, and Donaldson- and Gromov-Witten- like invariants of Fano 3- 
folds. It also allows us to define the holomorphic Casson invariant of a Calabi- 
Yau 3-fold X, prove it is deformation invariant, and compute it explicitly in 
some examples. Then we calculate moduli spaces of sheaves on a general K3 
fibration X, enabling us to compute the invariant for some ranks and Chern 
classes, and equate it to Gromov-Witten invariants of the "Mukai-dual" 3-fold 
for others. As an example the invariant is shown to distinguish Gross' diffeo- 
morphic 3-folds. Finally the Mukai-dual 3-fold is shown to be Calabi-Yau and 
its cohomology is related to that of X. 



1 Introduction 



This paper is a continuation of the ideas presented in [DT|, |T1|. There a formal 
picture was outlined in which the complex analogue of a real oriented n-manifold is 
a Calabi- Yau n-fold with a fixed holomorphic n-form playing the role of a "complex 
orientation", while a Fano with fixed smooth anticanonical divisor is the analogue 
of a manifold with boundary; the boundary being the (Calabi-Yau) divisor. I have 
since discovered this picture was known and used in low dimensions by the Yale 



school of Prenkel, Khesin, Todorov and others (see for instance [FK], | FKT |, |Kh|); 
we concentrate on the central, three dimensional theory of [pT| . The delay in publi- 
cation is due to a complete reworking of the deformation theory of Section 3. While 
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Tl| and earlier forms of this paper initially used an older version of [LT], then the 



derived category language of |BF|, here we present more elementary sheaf deforma- 
tion theory that allows us to use the more down-to-earth definition of obstruction 
theory in the published form of pL;T|| (without mention of the derived categories or 
T^-lifting of older drafts of this paper). This deformation theory is folklore but 
scattered and hard to find, and either highly abstract or not done in enough gen- 
erality for this application (namely in global families, as the complex structure on 
the manifold is allowed to vary, arbitrary order deformations and obstructions are 
considered, and the determinant is fixed). So we are forced to give a full account, 
for which the unpublished manuscript |^| has been useful. 

In Section 2 we review the classical Casson invariant and Chern-Simons func- 
tional, before describing their holomorphic analogues. We give two formulae for 
the holomorphic Chern-Simons functional, one in an algebro-geometric framework, 
the other illustrating the complex analogue of a manifold with boundary. We then 
discuss the holomorphic Casson invariant, and a version of it that so far has not 
been made rigorous but is reviewed as it motivates a number of calculations (and 
predicts the correct result for them). 

Section 3 discusses the technicalities involved in defining the invariant via gauge 
theory, and then tackles them using algebraic geometry, and the virtual moduli cycle 
theory of |LT], [BF|. The result required to apply this machinery to count stable 
sheaves is that the tangent-obstruction complex |LT| or cut-off cotangent complex 
|BF] (these are dual to each other) of the moduli space admits a certain two-step 
locally free resolution. We obtain one in all the cases we might hope for, namely 
whenever the higher obstruction groups ExiQ{£,£), i > 3, of the sheaves S vanish. 

This is the case, for instance, for ideal sheaves of curves in P^, so we obtain a 
virtual moduli cycle of the correct dimension in the corresponding Hilbert schemes. 
The result also applies to a Calabi-Yau 3-fold, of course, allowing us to make our 
definition. Finally in this section we give some examples, including one conjectured 
in DT | that is fitted into this scheme and worked out in full, by computing all 
reflexive sheaves of certain Chern classes on a (2,4) complete intersection in P^. 

Section 4 deals with bundles on K3 fibrations. The results here may be of interest 
to physicists, being the "F-theory" dual picture to the geometry of i^3-fibred 4- 
folds. Calculating moduli spaces of sheaves on threefolds is extremely difficult, and 
we are forced to consider only Chern classes satisfying certain constraints, though 
this allows us to work on a general K3 fibred Calabi-Yau X (without reducible or 
multiple fibres). Thus we obtain a calculation of the invariant in some cases (where 
it is 1), and obtain Gromov-Witten invariants of the "Mukai-dual" 3-fold in others. 
We then show this dual 3-fold is Calabi-Yau and determine its cohomology in terms 
of that of X. Finally we show that these results allow us to distinguish Gross' 
diffeomorphic Calabi-Yau 3-folds. 
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2 The holomorphic Casson invariant 



We begin by describing Taubes' version [Ta] of the Casson invariant in purely formal 



terms, ignoring such issues as the structure group and reducible connections; we shall 
be able to bypass these in the holomorphic situation by considering only bundles for 
which semistability implies stability (such as bundles with rank and degree coprime) . 

For us, then, the Casson invariant of a real 3-manifold M counts flat connections 
with structure group G on a fixed vector bundle E. Formally the curvature Fa of a 
connection A defines a closed one-form 

a^-^ tr(aAFA), aen^iadE), (2.1) 

on the space of connections £/. This is gauge invariant, and so descends to the space 
of gauge equivalence classes Fixing a basepoint Aq € £if this one-form is in fact 
the exterior derivative of a locally defined function, the Chern-Simons functional: 



CS{A) = j tr Qd^o" ^a + ^a ^a , 



A = Ao + a, 



which is independent of gauge transformations connected to the identity, and well 
defined modulo Z on In particular, at a zero of the one form, i.e. a flat connection, 
we see that the deformation complex of a flat connection is self-dual - this is the 
statement that the Hessian of CS" is symmetric - as then are its cohomology groups 
-ff*(adi?; A) = H^~^{8i,dE; A)* by Poincare duality. Therefore the virtual dimension 
of the moduli space of flat connections 



^(-l)^+Mimi7^(adS;A) 



i=0 
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is zero, and we could hope to count them. Formally, flat connections are the zeroes 



of the covector field Fa (2J) on i.e. critical points of CS, and we are trying to 
make sense of the Euler characteristic of the infinite dimensional space I^. 

This formal picture translates wholesale onto a Calabi-Yau 3-fold (which for us 
means a smooth, compact, Kahler 3-fold X with a trivialisation 9 G H^'''^ of the 
canonical bundle Kx — Ox)- Naively, we replace x hy z, d by d, Poincare duality 
by Serre duality, and integrating against the complex volume form ^ on X instead 
of against the orientation of M. 

So now we consider the space ^ of 9-operators (or "half-connections" ) on a fixed 

2 

C°°-bundle E X, and the closed one- form given by Fj^ : 



^ j tr(a A A 6*, a € S^^'^ad^). 



47r2 

Again this is gauge invariant and descends to the space of gauge equivalence classes 
Fixing a basepoint Aq ^ this one-form is the exterior derivative of a locally 
defined holomorphic function, the holomorphic Chern-Simons functional: 

CS{A) = ^j tr (^dAoa Aa + ^a Aa Aa^ A9, A = Aq + a, (2.2) 

which is independent of gauge transformations connected to the identity. The pe- 
riods under large gauge transformations are more complicated (and will usually be 
dense), but this will not concern us. All statements will be made "mod periods"; 
what we should really do is consider CS to be an element of some Albanese torus 



and formulate statements there, as discussed in |Tl|] , but this would take us too far 
afield. The zeroes of the one- form, i.e. the critical points of CS, are the integrable 
holomorphic structures on the bundle E (i.e. 5^ = F^'^ = instead of (f = Fa = 0), 
and the deformation complex of a holomorphic connection is self-dual - again the 
statement that the Hessian of C5 is symmetric - as then are its cohomology groups 
H'^''-{adE;A) ^ H°'^-'{adE;Ay , by Serre duality and the fixed trivialisation 9 of 
the canonical bundle Kx- Therefore the virtual dimension of the moduli space of 
holomorphic bundles 

3 

^{-iy+^dimH°''{adE;A) 
1=0 

is zero, and we could hope to count the bundles to formally compute the Euler 
characteristic of 

Of course to get some kind of compact space of objects to count we have to 
consider either Hermitian- Yang-Mills connections in the gauge theory set-up, or 
stable holomorphic bundles in algebraic geometry; this is in some sense most of 
them. 
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Similar holomorphic analogues of all the main gauge theories also exist [DT] 



| T1 [ and have also now been studied by physicists | AOS| , [ BKS |, and there is work 



of Tyurin [ry2| on related topics. Also, formally manipulating the holomorphic 



analogue of the Chern-Simons path integral using CS (|2.2| ) gives a holomorphic 
linking number for complex curves in a Calabi-Yau manifold Jri| but since I have 
discovered the more professional treatment of |FKT| , | KR |, here we concentrate 



solely on defining and calculating the holomorphic analogue of the Casson invariant. 
Firstly, however, we give two formulae for CS" to make it more familiar and further 
illustrate the holomorphic analogy. 

The first is in an algebro-geometric spirit, and well known on complex curves 
as Abel-Jacobi theory: fixing a complex curve S the appropriate Chern-Simons 
functional computes holonomy of 5-operators A = d + a on a, topologically trivial 
line bundle, for simplicity. It is 

s 

modulo periods. Again these periods are dense and the function is only well defined 
modulo a discrete lattice when considered as a function of all us at once, as an 
element of the torus 

(i/i'0)7i7i(S;Z). 
In this case we also have the alternative formula 



a f\ui = i UJ modulo periods, 

where 7 is a path connecting the points that are the zeroes and poles of a section 
that is meromorphic with respect to the holomorphic structure d + a. That is, if 
the holomorphic structure defined by a on the line bundle corresponds to the divisor 
X^Oi(pj), with G Z and G S, then = J^'^iiPi)- The principle of the result 
is that the Euler class of the line bundle is represented holomorphically by both the 
divisor, which is ^7, and the curvature da, and d and d are adjoints. 

In a special case there is an analogous formula on a Calabi-Yau 3-fold, for a rank 
2 holomorphic bundle E. Similarly the principle is that dCS = pi f\ 9 and, when 
det E is trivial, pi = C2 represents the Euler class of E, so we are interested in a 
homology class A, the analogue of 7, with boundary the zero set of a section. (Here 
and below we use CS to denote both the functional (|2.2| ) and the integrand.) 



Proposition 2.3 [Tl] Suppose that Aq and A = Aq -\- a are integrable (F^'"^ = Oj 



d-operators on E with trivial determinant, and {E,A), {E,Ao) admit holomorphic 
sections s, sq, with transverse zero sets {s)q, (so)o- Then CS defined by may 
also he described as follows. As the zero sets are homologous, write [s)q — [sq)q = 9A 
for some singular 3-chain A. Then, modulo periods, CS{A) — CS{Aq) = J^O. 
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The second formula is the complex analogue of the classical formula for the 
Chern-Simons functional of a connection A on a bundle £^ — > M on a real 3-manifold: 
bound M by a 4-manifold N and extend {E, A) to a bundle and connection (E, A) 
on N. Then 

CS{A)-CS{Ao)= [ pi(A)-pi(Ao), 

where pi(A) = (l/47r^) tr i<A A Fj^ is the Chern-Weil differential form representing 
the first Pontryagin class of E. This can be rephrased in a way that will make the 
complex analogy more apparent in terms of the long exact homology and cohomology 
sequences of the pair {N, M) , by the commutative pairings 

[pi(A)-pi(Ao)] 

H'^{N,M) 
Hi{N,M) ^ 0. 

[TV] 

That is, the fundamental class of M is in the image of the lower map, coming from 
the fundamental class of A^, so to find fj^j CS{A) we can map the class [05(^4)] into 
H^{N,M) and evaluate on [N] to give the result. 

The holomorphic analogue replaces the exact sequence of the pair (A^, M) by the 
sheaf cohomology sequence of the pair {Y,X), 

0^ Ky ^Oy ^Ox ^0, (2.4) 

in the case that X is an anticanonical divisor in a 4- fold Y. Here then we think 
of X as being bounded by the "Fano" Y; we use the term Fano loosely to mean a 
variety Y with a section s of its anticanonical bundle Ky^ with a smooth zero set 
X, which is its "boundary" - it is Calabi-Yau by the adjunction formula. In fact 
about a point of X C Y, choosing a local coordinate z whose zero locus is X, to 
leading order = uniquely defines a holomorphic 3-form 6 on X ( jGH[ | p 

147). Then we obtain 

Theorem 2.5 Suppose that the Calabi-Yau 3- fold X is a smooth effective anti- 
canonical divisor in a 4-fold Y defined by s G H^{Ky^). If E ^ X is a bundle that 
extends to a bundle E ^ y, then for a d-operator A on E, let A be any d-operator 
on E extending A. We have, modulo periods, 

CS{A) = ^^l^trF'/AFrAs-\ 



[CS{A)] 
Hs{M) 

m 
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Proof Notice that i^^(Oy) = H°{Ky)* = 0: if t G H^{Ky) then s.t is a holomorphic 
function on Y vanishing on X, thus t = 0. So the sequence ( |2.4D gives us the fohowing 
commutative diagram of Serre duaUty pairings, 

[CS{A)] ^ [pi(A)-pi(Ao)]As-i 



[1] ^ [1] 



(2.6) 



(the first pairing is by integrating against 6) since the upper map takes a holomor- 
phic (0, 3)-form on X, extends it to a C°° form on Y, and takes 9( • ) A of the 
result. Setting CS{Aq) = JyPi(Ao) to fix constants gives the result, q.e.d. 

Just as the real case CS{A) = J^pi(A) can be proved directly by Stokes' the- 
orem, the above amounts to an application of Stokes' theorem and the Cauchy 
residue theorem, hence reducing dimensions by two, as also observed in |Kh|. If 



1^5 (X) denotes a small tubular neighbourhood of X CY then, by Stokes' theorem. 



Pi(A)As-^= / d{CSiA) As-^) = \im / CS{A)As^\ 

Y Jy <5-*0 J dug 

which can be integrated first over the fibres of the circle bundle dus — > X, and then 
along X, by Fubini's theorem. As s^^ ~ 2^^^' integration over the fibres gives, 
by the Cauchy reside formula, f-^ CS{A) A 6* in the limit of 5 ^ 0. 

Hence, just as d is adjoint to the boundary operator d in real geometry, d{ ■ ) As^^ 
is adjoint to this complex boundary operation of taking the anticanonical divisor (s)q 
with its induced complex volume form. An application of this to holomorphic linking 



is given in | KR | - the 9-Green's function for the current represented by a complex 
curve (weighted by a holomorphic one-form) is represented, as a current, by any Fano 
surface containing it as an anticanonical divisor. Thus integrating this over another 
curve (against a one-form), to give Atiyah's holomorphic linking number, is the 
same as intersecting this second curve with the complex surface, and weighting in- 
tersection numbers by ratios of the holomorphic volume forms at intersection points. 



Finally we mention briefly the holomorphic analogue of Casson's original ap- 
proach to counting flat connections by splitting a 3-manifold M (in fact a homology 
sphere) across a Riemann surface S, 

M = Ml Us M2. 

The orientation of M induces a symplectic structure on E, and so one on (the smooth 
locus of) its moduli space of flat connections A4j]. Then those connections on S 
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that extend to flat connections on Mi form a Lagrangian submanifold in A^Ej the 
image of the restriction map ^AMl M.^. Similarly for AiM2- These are both of 
half dimension so we expect them to intersect in a finite number of points - the flat 
connections on S that extend to both Mi and M2, i.e. the flat connections on M. 
Casson overcomes the technical difficulties to define just such an invariant, counting 
(one half of) all the flat connections except the trivial one. 



Although not yet rigorous there is a holomorphic analogue of this |DT], [Kh] 



following work of Tyurin [Tyl]. We review it briefly because it motivates some 
examples and is verified in all of them. Our complex analogue of gluing across a 
boundary is to take two Fano 3-folds Xi with a common anticanonical divisor S, 
and form the normal crossings space 

X = XiUs X2, 



1} 



which is a singular Calabi-Yau. The (singular) holomorphic volume form on X, 
with poles along S, induces a complex symplectic structure on the surface S (this 
is just the adjunction formula) and so on its moduli space of (stable) bundles ^As 
|Mul|. Then those bundles on S that extend to holomorphic bundles on Xi form 



a complex Lagrangian submanifold given by the restriction map Aixi -Ms (at 
least where this is defined, i.e. where stability is preserved on S); similarly for X2. 
As before intersecting these 

Mxr n Mx, (2.7) 

in Ms we expect to get a finite number of holomorphic bundles that extend to both 
Xi, i.e. bundles on X. In the examples we consider X will be smoothable and the 
number of bundles will be preserved on smoothing to give the holomorphic Casson 
invariant of the smooth 3-fold. 



3 Virtual moduli cycles 

To count (stable) holomorphic bundles on a Calabi-Yau 3-fold, there are two things 
we require of the moduli problem - compactness and transversality. In gauge theory 
such results are easier in lower dimensions. In two real dimensions moduli of stable 
bundles are both compact and of the right dimension. In three dimensions this can 
be achieved after a perturbation [ [Tal (leaving aside problems with reducibles) for flat 
connections. In dimension four we need both perturbations to achieve transversality, 
and a compactification to take account of the non-compactness caused by conformal 



invariance ]DK|]. Just recently we now have the results of Tian |Ti| in higher dimen- 



sions, proving just about everything that one would like to be true, giving a natural 
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analogue of the Donaldson-Uhlenbeck compactification of four dimensions. For a 
Kahler 3-fold this involves ideal instanton singularities along holomorphic curves in 
the 3-fold, but also some codimension 3 singularities that are harder to deal with. 

What is missing, however, is a transversality result. Staying within the confines 
of algebraic Calabi-Yau manifolds we cannot hope to get a moduli space of the correct 
dimension; there is no result along the lines of Donaldson's generic smoothness result 
for moduli spaces on algebraic surfaces. One reason is that, in the rank two case 
for instance, relating the deformation theory of a bundle to that of a zero set of a 
section via the Serre construction, points are unobstructed on a surface but curves 
can be obstructed on a 3-fold (there is a fuller discussion of this and other such 



issues in [ri|, and an example in T2|). 

So we would like more perturbations. There is an elliptic perturbation of the 
Hermitian- Yang-Mills equation valid on any almost-complex symplectic manifold, 
which I learnt from Donaldson: 

T-i0,2 



A 

iAF^^ = XI. 



(The problem is that the Hermitian- Yang-Mills equations appear over determined, 

— Q 2 

but are not because of the Bianchi identity dF^ = on a Kahler manifold. This 
can be formalised by introducing the (0,3)-form u as above, and then d*u vanishes. 
It need not be zero in the almost complex case.) 

It seems that Tian's work |T| should also apply to these equations (with the 
singularities now along pseudo-holomorphic curves and at points) so long as we can 
get a bound on ||Fa||^2 similar to that given by characteristic class formulae in the 
integrable case. A Weitzenbock formula shows that this is the case if, for instance, 
the scalar curvature is everywhere positive. Thus, at present, this would work best 
for Fano 3-folds and the Calabi-Yau case is borderline. Either way a transversality 
result, proving that for generic almost complex structures the moduli space of solu- 
tions is of the correct dimension, seems a long way off, as does understanding the 
codimension three singularities. 

So we turn to algebraic geometry where we have the now standard compactifi- 
cation of the moduli space of stable bundles by semistable sheaves, due to Gieseker, 
Maruyama and Simpson. This moduli space will invariably be singular and of too 
high a dimension, but it is often clear what the contribution of a particular com- 
ponent to the "number of bundles" should be - the Euler number of its cotangent 
bundle in the case it is smooth, two if it is a scheme-theoretic double point, etc. 
In the general case there is the machinery of [ [LTj ], |BF| to produce a "virtual mod- 



uli cycle" of the correct dimension (zero, for us) inside any moduli space satisfying 
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certain conditions; we briefly outline the picture. 

Suppose a variety M (which will eventually be our moduli space M) sits inside 
a smooth ambient n-fold Z, cut out by a section s of a rank r vector bundle E ^ Z. 
Then the "virtual dimension" of M is [n — r) ~ the dimension it would be were 
s transverse. If it is not but, for instance, s lies in a subbundle E' <Z E and is a 
transverse section of then it is clear the "correct" (n — r)-cycle we should take is 
the Euler class of the cokernel bundle E/E' over M ~ this is homologous to the zero 
set of a transverse perturbation of s if one exists. In the general case dealt with by 



Fulton-MacPherson intersection theory [Fu|, s induces a cone in E\m, which can be 



thought of as "s made vertical", i.e. the limit of the images of As as A — > oo. We 
may then intersect this with the zero set M inside E to get a cycle in M (whose 
image in Z represents the top chern class of E, as required). 

The point here is that we worked entirely on M and not in the ambient space Z, 
and so we might hope the method is applicable to moduli problems where the am- 
bient space Z does not exist. Instead the deformation theory of the moduli problem 
often gives us the infinitesimal version of (Z, E^ s) on M, namely the derivative of 
s, yielding the exact sequence 

^ TM ^ TZ\m ^ E\m ^ ob ^ 0, (3.1) 

for some cokernel ob which in the moduli problem becomes the obstruction sheaf. 
In the general case we require a global version of this, namely a two term locally 
free resolution 

^ Ti ^ ^1 ^ ^2 ^ ^2 ^ 0, (3.2) 



of the tangent-obstruction functors to be introduced presently ( |3.26| , p.28| ). 

Here Ei and E2 play the roles of TZ\m and in the above motivation (these 
last two have the same fibre rank at each point of A4, and hence are locally free) 
and have difference in ranks equal to the virtual dimension of the moduli problem. 



It is shown in [LT|, |BF| that such data on M is in fact sufficient to obtain a cone in 
the vector bundle Ei which can be intersected with the zero set ^A to give a virtual 
moduli cycle with the correct properties. The precise statement is given below. 



First then, we need to develop the necessary sheaf deformation theory. An 
earlier version of this paper used the cotangent complex approach of |lBF| , and 
Lehn's description Q of equations cutting out the Quot scheme to calculate it; the 
deformation theory here is more classical and natural, even if it is a little longer, 
dealing with higher order deformations. 

An excellent reference for the sheaf theory we use is p^H |; we shall assume fa- 
miliarity with Gieseker stability (here always referred to just as stability), slope 
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stability, the fact that 

slope stable ^ stable =^ semistable =^ slope semistable, 

and that for rank and degree coprime the circle is completed by slope semist ability 
implying slope stability. Also refer to [ [HL| for the fact that for either form of stability 

(slope) stable ^ simple, 

i.e. the only endomorphisms of the sheaf are the scalars C . id. Finally, stable 
sheaves are pure, i.e. torsion-free on restriction to their support. 

3.1 Sheaf deformation theory 

In this section all schemes will be complex and quasi-projective, and all sheaves 
coherent. We begin by recalling the definition of the trivial thickening 

S*n (3.3) 

of a scheme S by a coherent sheaf of O^-modules n. Namely make the O^-module 
05©n into a sheaf of rings by stipulating that = 0, giving the trivial ring extension 
Os * Ti; the associated scheme is 5 * n. Then sheaf deformation theory is built on 
the following standard result. 

Lemma 3.4 Let n be a sheaf on a scheme S, and let S * n be the trivial extension 
of ( [3.4 ) above. Then deformations of a sheaf £ on X x S, fiat over S, to a sheaf on 
X X (S * n), fiat over S * n, are in 1-1 correspondence with Ext^x5(£', ^ ^X) n). 

Proof Given such a deformation tensoring with the sequence 

^ n ^ Os*n ^ Os ^ 0, (3.5) 

gives a sequence, exact by flatness, 

O^£-0n^Jf^£:^O (3.6) 

on X X [S * n), using the fact that the left and right hand terms of ( |3.5| ) are O5- 
modules. The sequence (|3.6|) defines the class in 'Ejyit\y^g{£,£ (g) n). 

Conversely such a class gives a sequence of 0xx5-niodules ( |3.6[ ), which defines 
an Ox x(s*n) "module J- since there is an obvious action of O5 * n on .F: the n- action 
is given by projecting T ^ £ and tensoring with n, mapping to £" (?) n C J^. Flatness 
over X X (5**11) also follows from the sequence ( |3.6D and the following Lemma, q.e.d. 
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Lemma 3.7 Let S C Y be a subscheme with ideal n C Oy such that = 0. Then 
an Oy - fnodule T is flat overY if and only if J^®n !F is injective and J^\s = J^<SiOs 
is flat over S. 

Proof We must show ( [|Hal[| III 9.1 A) that T J !F is injective for any ideal 
J C Oy. Given such a J, we have an exact sequence 

where J' is annihilated by n so is naturally an ideal in O^. Thus the diagram 



i 

«) ( J n n) ^ J ^ J^ls J' ^ 

i I \ 

^^0n ^:F\s ^0 

is exact by our hypotheses. To show the central vertical map is injective, then, 
it is sufficient to show that the first vertical map is injective. But this map is 
J^\s {J Cin) !F\s ® n, through which !F\s ^ [J Hn) !F\s factors, and !F\s is 
assumed flat so this last map is an injection, q.e.d. 

Next we consider obstructions to deformations. We will repeatedly use the fol- 
lowing set-up. 

• Let C y C li all be schemes over S, and denote the ideals of 5 C y, S C 
Yi, y C li by n, m and respectively. Assume also that m. = 0, giving 
an exact sequence of Oy-modules 

O^J^^m^n^O. (3.8) 

Notice we do not assume that Y = S *n this time, or even that = 0. 

Given a sheaf £ over X xY (flat over Y and restricting to Sq over S) we get an 
exact sequence 

O^S(g)n^S^£o^O, (3.9) 

giving a class e G Ext^x5(£'o, f CSn) (we will consider all terms as O^-modules using 
the projections y — > 5, Yi ^ S, so all Exts will be over X x S from now on). 
We would like to lift this to an on Yi to give a sequence 

O^S^m^J'^So^O, (3.10) 
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(here JF m = £^ m since m.J^ = 0) defining / e Ext^(fo, f m). / is a lift of e 
in the sequence 

Ext^(£:o,^ (»m) ^Ext^(£:o,£:8)n) Ex.t'^{£o,£o (g) J^), (3.11) 
obtained by applying Horn {Sq, . ) to the sequence 8® (P-8D: 

0^£:o«)=^^£'(»m^£'(g)n^O. (3.12) 
(Exactness follows from the flatness of £ over Y.) 

Proposition 3.13 £ over X x Y as above extends to a sheaf T over X x Yi, flat 
Yi, if and only if there is a lift f G Ext^(£'o, CSm) ( 3.10 ) of e G Ext^(£'o, £" (8>n) 



over 



(^^), i.e. if and only if de G Ext^(£'o,£'o <8> J^) in the above sequence ( 3. 11]) is 2;ero. 



Proof We are left with showing that the existence of a lift / of e gives such an JF. 
/ gives a sequence ( ^.91 ) of Os-'^odules lifting ( 3.1C| ), and so a diagram 



1 t' II 

^ f (g) n — *- £ — *- £^0 ^ 

1 \ 

0. 

Here JF is an Os-module, and we would like to make it an Oy-^-module, where 

Oy^ = O5 © m via the maps S Yi. So we define the action T ® m ^ T hy 
i o (vr (8) id) in the above diagram. 



Flatness of J- over Yi then follows from Lemma 3.7 on noting that J-\y = £" is 
fiat over Y , and J^^ = since C m and m . = 0. q.e.d. 

We note in passing that d is cup product with the element e' G EyX^{£®x\, £q<^^) 
defining the extension ( 3.12| ), so £ extends to T if and only if e'Ue G Ext^(£'o, £'o'X'=-^) 



IS zero. 



The trace map 

We have now pretty much found the tangent-obstruction (in the sense of [ |LT| ) com- 
plex of the moduli problem for stable sheaves, as we shall see below. Unfortunately 
we are more interested in the moduli problem for sheaves of fixed determinant, for 
which we need the machinery of the Mukai-Artamkin trace map (in a little more 
generality than | Mul[ , [Ar[ , for higher order deformations). 
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Given a coherent sheaf on a quasi-projective scheme X and an affine open 
cover lA = {Ui : i = 1, . . . , n} of X, denote the Cech complex by 




io<...<ip 



with the usual Cech differential 5 : & ^ ( pal| III 4.1). This computes the 

sheaf cohomology of and as the construction is functorial in it can also be 
applied to a complex of sheaves T' to give a double complex whose associated total 
complex has cohomology the hypercohomology EI*(^*). 

If X is smooth then any sheaf 8 has a finite locally free resolution ^ f ^ 0, 
and the trace map is defined as follows (by standard arguments it will be indepen- 
dent, up to quasi- isomorphism, of the choices U and E'). 

Given any sheaf T, form the complex T-Lom'{E'^E' ®Z) with T-Lom}{E',E' (8) 
Z) = ®jnom{E'J ,E'+i (g) I) and differential # = d'+^ o cj) - {-Ifcp o for G 
Tiom^E^ , E^~^^ ®Z). This admits cochain maps 



That these are cochain maps follows from the easy computations tiod = and 
d o id = 0. 

Notice that tr o id = o id^;. = X;(-l)* rk (S*) = rk(f), so, for 

rk {£) > 0, we have a splitting 



nom'{E',E' (gl) ^ I, 



(3.14) 



with the upper map given by 



tr = ^{-ly tr^(g)idj 




nom'{E',E' J) = nom'o{E',E' 01) © X, 



where Tiom^ is the kernel of tr. 



Thus (3.14) induces cochain maps between Cech complexes 



C'{nom'{E',E' ®Z)) ^ C'{1), 



inducing maps tr and id on cohomology 



Exf(£:,f g)!) ^ H\l) 
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such that tr o id = ik£. Thus for rk(£) > 0, there is a sphtting Ext*(£',£' ^I) = 
Ex.ti{£,£®I) e with Exto the kernel of tr. 

To work towards showing that taking the trace of deformations and obstructions 
of a sheaf gives the deformations and obstructions of the determinant of the sheaf, we 
first need this (rather elementary) fact for locally free sheaves. Of course, phrasing 
the deformation theory of holomorphic vector bundles in terms of connections or 
transition functions this is simple; the work is then in showing the deformation 
theory coincides with the abstract sheaf deformation theory of the last section. This 
involves simple but very large computations with transition functions as in 
Here we prefer to work directly with our definitions above; this then makes the 
proof below a little long, but the reader could take it on trust. 



Proposition 3.15 Take S* C 1^ C Yi to be as in (3.8). Suppose we have a rank r 
locally free sheaf E on X xY , a flat deformation of Eq on X x S giving an extension 
e G Ext^^Ef), E (8) n) ( [g.^ j. Assuming first that = 0, then the extension defined 
similarly by the determinant hJ'E is 

tr(e) G H^{n) = Ext^(A''So, A^'Sq n). 

Likewise, for any n, given the obstruction de G Ext^ (£^o j -E'o ® ^) of Proposition 
3.13^ to extending E over X xY to F over X xYi (flat over Yi), the obstruction to 



extending A^E is given by 

tr(5e) G H^iJ^) = Ext2(A^^o, A^'^o ® 

Proof We begin by giving explicit descriptions of e and the obstruction de. 
Applying Hom(£'o) • ) to the extension (of OxxS-modules) 

O^E(g)n^E^Eo^O (3.16) 

gives a connecting homomorphism Hom(£'o) Eq) — > Ext^(£'o) E0n) under which the 
image of id^;^ is the extension class e G Ext^(£'o5 E ®n). 

So consider the exact sequence of complexes below given by applying the exact 
functor 7iom(EQ, . ) (recall that £"0 is a locally free O^-module) to the sequence 
( p. 16 ) and taking Cech complexes: 



^ C'{nom{Eo,E(E)n)) ^ C' {nom{Eo, E)) C'{nom{Eo,Eo)) 0. (3.17) 

idgg gives a closed element of C^{7iom{EQ, Eq)) (i.e. a global section). Lift this to 
some a G C^{Tlom{EQ, E)), giving, for each U, V in some affine open cover U of X, 
au, ay such that over U (1 V , ay — ajj ='■ i-{eunv) defines the (closed) element 

e G C^{nom{EQ,E ®n)) 
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that represents e € Ext"'^(-E'o) E (g) n). 

To identify de we use the similar exact sequence of Cech complexes 

^ C'{nom{Eo, Eo ® J^)) ^ C'{nom{Eo, E®m)) ^ C'{rLom{Eo, E^n)) ^ 0, 

associated to the sequence ( p.ll ). 

That is, choose lifts bjjv £ Hom^y(£'o) E iSim) of euv Then over U nV (1 W , 
buv + bvw + bwu = '^i{de)uvw) defines {de)uvw ^ ^omuvwiEo,Eo (g) J^) giving 
the obstruction class de € Ext^(£'o) Eq (g) J^). 

We now use these explicit calculations to repeat the above exercise on the induced 
deformation of determinants 

0^ A^'E^n^ A^'E ^ A^'Eo^O. (3.18) 

Here for A'^E we have taken the rth exterior power of E as a sheaf of x xY -iTiodules 
{not as a sheaf of 0xx5-modules) but then we consider the result and the above 
sequence as OxxS-niodules. Thus we have the analogue 

O^C'{nom{A''Eo,A''E^n)) ^ C'{nom{A''Eo,A''E))^C'{nom{A''Eo,A''Eo))^0 
of (|3T7| ). 

Use A^'au G B.omu{A''Eo,A''E) to lift idA^ij^ = A'^id^o G Romu{A'' Eq, A'' Eq). 
Over U nV, A^'au - A^ay = i{euv) defines euv G Hom^/y (A''£'o, A''£' ® n), the 
extension class e G Exti(A'"£;o, A'"^ ®n) = H^{n) of 

Over U nV there is a splitting of (|3.16|) induced by aij: 

E^{E(S)n)eEo, (3.19) 

with respect to which au = © id^;^, ay = eupiV © id^o! and (|3.18|) splits as 
A'''E = {A^E (8) n) © A^Eq (we are omitting some \ur\V s for clarity). 

So over [/ n F, euy = A^{euv © id) — id, euw = A** [euw © id) — id with respect 
to this splitting, while eyw = {^uw © id) — A^{ejjv © id). 

For the first part of the Proposition we want to show that, for = 0, these es 
are (the images under l of) the traces of the corresponding es (i.e. euv = ^"^^i^uv)-, 
etc.). But this is clear from the expansion 

A'Xidso ©e) = idArijg +tr[7(e) + 2iTu{A^e) + . . . , 

for e : Eq ^ E n, where tiu{^^&) is a map of the right kind, i.e. an element of 
{A^EqY © A^E © n, on defining tiu by the composition 

A^El © A^E © n ^ A'' El © A^^o © n ^ n ^ A^E^ © A'^^o ® n ^ A^'E^ © A^E © n 
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of the projection, trace, identity and K^ajj maps respectively. This is just a glorified 
version of the expansion of the determinant in terms of the elementary symmetric 
polynomials of e, but over the ring Oy. For = it is just the usual trace, 
independent of a[/, since any two as differ by something in the ideal n. In this case 
all the higher order terms in the above expansion become zero anyway leaving just 
tr(e), giving the required result. 

To identify the obstruction we pick the obvious lifts of the es. That is, over 
[/ n y in the splitting ( |3.19| ), euv = trc/Cec/y) + 2 tiu{h?euv) + . . . , so we set 
(Sjjv = i'^u{buv) + 2 ivjj{h?bjjv) + . . . . (tr^/ is defined by the same formula as before 
but with m replacing n. Then f3uy is actually skew symmetric with respect to U 
and y, so is well defined, though it takes a calculation relating tiu and try to check 
it.) 

The class of the obstruction we are seeking is given, on [/ n ^ fl W , by 

Puv + (ivw + fiwu = trc/(6c/y) + iTv{hvw) + ^"^wQ^wu) + ■■■ ■ (3.20) 

We claim that only the terms with a linear dependence on the 6 s are non-zero up 
to coboundaries; this can be checked by a large local calculation or the following 
cheat. We know that hyw = —huv — bwu + h where i lies in the ideal (i is of 
course L{de)uvw)-) If we had chosen the different lift —huv — bwu of eyw then 
( p. 20 ) would of course give zero up to coboundaries. Thus ( p. 20 ), considered as a 



polynomial in i, has zero constant term, and any term of order > 2 (in the 6 s or 
i) involves i multiplying something in m, which vanishes since m . ^ = 0. Thus the 
obstruction class is the just the linear part of ( p. 20 ), which is tT{{de)uvw) since 
buv + bvw + bwu = ''i{de)uvw)- q-e-d. 

Similar explicit calculations over patches of a cover U also give the standard 
results that, for line bundles Lj with deformation and obstruction classes e(Lj) and 
d{e{Li)) as above, we have 

ei^iLi) = G H\n), and 9(e(®iLi)) = 9(e(Li)) G ^'(n). 

i i 

(3.21) 

Also recall that for any sheaf £ with a finite locally free resolution E', the 
determinant of £ is defined to be the line bundle 

det^ = (g)(detS*)(-^)\ (3.22) 

i 

which is independent of the resolution. 
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So as before let X he a smooth quasi-projective variety and let 5" C 1^ C li be 
as in (|3.8| ). Fix a sheaf £ on X xY that is flat over Y and restricts to £"0 on X x S. 
Then since X is smooth, £ has such a finite locally free resolution E' that restricts 
on X X S, by flatness, to a flnite locally free resolution Eq of £0. 

Theorem 3.23 In the above situation, denote by e G Ext^(£^Oi^ ^ Ti) o,''^d de G 



Ex.t^{£o,£o^ ^) the deformation and obstruction classes of £ ( [3-4 3.1^ . Then the 



obstruction class of det£ is tr((9e) G H {^) = Ext (det £"0, det £^0 ® -^), md, if 
V? = 0, the deformation class o/det £ is tr(e) G H^{^) = Ext^(det £"0, det £"0 (8) n). 

Proof Again we explicitly chase Cech cocycles representing the extension and de- 
formation classes. The exact sequence — > n — > OxxY ^XxS gives the exact 
diagram of resolutions 



0- 



E' 



0- 



■£' 



E' 



■£■ 



E'o- 

i 

-£o- 
1 

0. 













In turn this gives an exact sequence of Cech complexes 

^ C'{nom'{E^,E' 0n)) ^ C'{nom'{E^, E')) C'{nom'{E^, E^)) 0, 

where the first complex computes Ext^(£'o,£' ® n), etc. id G {Tiom' {Eq, Eq)) is 
e^idi with idi G C^{rLom^{El,Ei)). Lift this to a G C^{nom'{E^, E')) 
with ai G C^{nom^{Ei,E')). 

We now apply the total differential d + 5 io a, with d the differential on Tiom' 
and 5 the Cech differential, keeping track of degrees. We get 



(S + d)a = L{eiei + ®ifi), 

where e^ G C\nom^{El, E' ^n)) and fi G C^(nom{El, E'^^ ®n))®C^{nom{El 
E' (g) n)). Thus L{ei) = 6{ai) (and = d{ai)). 

So ©jCj + ©j/j is closed under d + 6 and represents the class e G Ext-'^(£'o, £' 
n) of the deformation £, whereas is closed under 5 and represents the class in 
Ext^(i?Q, ©n) of the deformation E\ Suppose that = so that EiS>n = EQ(S}n, 
etc. and tr is defined. Then since tr only acts on TlomP components of the complex 
( p. 14 ), we see that 



tr(e) = tr(©iei + ®ifi) = tr(©,ei) = ^^(-l)' tv{e{E')) = ^(-1)' e(det E'), 
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by Proposition 3.15 . By ( |3.21| ) this is e(det£'), as required. 



To deal with obstructions use O^J^— >m^n— >0to give the exact diagram 
— ^ E^(g)J^ — ^ E'(g)m — ^ E' (g)n — ^ 

III 

— £o'S)J^ ^ £®m *- f (g) n 

III 
0, 

giving the exact sequence of Cech complexes 

^ C'{Hom'{E'f^,E'Q®Jf)) ^ & {Horn' {E'q, E' ®m)) C'{nom'{E^, E'm)) 0. 

Lift our class e = ©jCj + (Bifi to ©j6j + ©jQ in C' {TLom' {Eq, E' (g) n)), and apply 
5 + d to give, in different degrees, 

®i5{bi) + ®i{5{ci) + d{hi)) + ®id{ci). 

This is L{de), by definition, with the component of de in C^iTLomP) being a sum ©jOj 
of terms such that t(oj) = 5{hi). Thus Oj is, by the definition of hi, the obstruction 
de{E'^) to the extension of E^, and since tr only acts on TlomP components of the 
complex (|3.14| ), i.e. on only the (Bide{E^) parts of de, we have 

tT{de)=tr:{eide{E')) = Y,i-'^yHde{E')) = "^{-iydeidetE'), 



by Proposition 3.15. But by (p.2lD this is 5e(detf), as required, q.e.d. 



We are finally in a position to find the "tangent-obstruction complex" of our 
moduli problem, as defined in |LT| (though our moduli problem is contravariant, 
not covariant). 

Fix a smooth quasi-projective scheme X and Chern classes q € H'^'^(X), and 
consider the moduli functor A4 that assigns to any scheme S the set of isomorphism 
classes of sheaves £ on X x S, flat over S, whose restriction to each fibre is stable 
and has Chern classes Ci. Here two sheaves are considered isomorphic if they differ 
by tensoring with a line bundle on 5. This moduli functor has a coarse moduli 



space which we also denote hy A4 = A4{X, Cj) |HL|, such that any sheaf £ as above 



induces a morphism / : S ^ A4. Similarly for the sub-moduli problem Ml for 



sheaves of fixed determinant L ( 3.22 ), with its moduli space A4l C A4. We shall 
call this standard data: 

• S" an affine scheme, 
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£q on X X S, stable on each fibre of p : X x S ^ S and flat over S, 

Chern classes Ci{£o) £ H'^^{X), a rank r{£) £ H^{X), and a line bundle L on X 
with ci{L) = ci, 



The corresponding classifying morphism / : S ^ A4 = M{X, Ci), 

(3.24) 



An 05-module I 



Here we have also included the arbitrary 05-module 2, along which we will consider 
deformations. 

Definition 3.25 [ p^T| Given standard data (3.24) above, the tangent functor of 
the moduli functor assigns to each I an Os-i^^odule Tg^iZ) such that the set of sheaves 
on X X [S *Z) restricting to Sq on X x S is isomorphic to 

rsiTlii)). 

We also require that given Eq on X x S2, a morphism of schemes f : S*! — > 5*2 and 
a homomorphism f*l2 — > Ii induce a canonical homomorphism 

f*Tl{i2)^rj,,^{i,) 

compatible with base-change as in [|LT|. 



Denote by £xtp the right derived functors of Tionip = p^, Tlom, where p : X xS ^ 
S. The functoriality and compatability with base-change properties will follow from 
those of Sxtp in our case. For S affine, so that of any O^-module vanishes, the 
Leray spectral sequence shows that Ext^ ^ ^ (£"0 , £"0 ^ p*I) = Tg{8xtp{£Q,£Q ^ p*Z)) 
and similarly, for its trace- free counterpart, Ext^^^(£'o, £'o<K'P*2r)o = ^s{£xtp{£o, 
P*I))q. Thus we have, by Lemma 3.4 and Theorem 3.25 , 



Proposition 3.26 The tangent functor for the above sheaf moduli problem A4 as- 
signs to any Os-module X the Os-niodule 

TliI)=£xtl{£o,£o^P*I). 

For the moduli functor Ml ihe same applies to the trace-free Os-module (7^"Jj)o(X) = 
£xtl{£o,£o(S>p*I)o. 
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Definition 3.27 [LT| Given standard data ( 3. 2J^ as above, an obstruction sheaf for 
the moduli functor is an Os-moffu/e 7^ satisfying the following conditions. 

Let S CY cYi be as in (3.8), and take a sheaf £ over X xY that is flat over 
Y , stable on the X-fibres, and restricts to Sq on S. Then there is an obstruction 
class 

oh {£,Yo,Yi) GTsiTl^y) 

whose vanishing is necessary and sufficient for there to be an Oyi -flat extension of 
£ to £i over X xYi. This should be functorial and canonical under base-change (as 
in [LT] 1.2, but with contravariance replacing covariance). 



Theorem 3.28 Given any affine scheme S and a sheaf £q on X x S as above such 
that dimExt*(£^o|xsi'fo|xs) is constant for all s ^ S and i >3, the sheaves 

^0 = ^xtl{£o, £o) and {T^^^)q = £xtl{£o,£o)o 

are obstruction sheaves for both M and Ml- 



Proof Theorem 3.13 shows that 



Ob {£, Fo, Yi) := d{ee) G Ext^^xsi^o, So ® P*^) 

defines an obstruction class, and Theorem ^.23| shows that in fact it lies in the 
subspace Ex.t'j^^g{£o,£o ®p*^)q since deformations of the line bundle detf are 
unobstructed (PicX is smooth since X is smooth). By the Leray spectral sequence 
Extjf xsC'^O) £o'^p*^) = Ts{£xtp{£o,£o ^)) (and similarly for the trace- free Exts, 



for which the rest of the proof is also similar and is omitted) 
There is a spectral sequence Torj{£xtp{£o,£Q),J^) 



{£o,£q(S)P*^) (by 



standard nonsense and the flatness of £o so that Tovj{£o,p* J^) = 0, j > 0). By 



base-change (e.g. | BPS[| ) the hypothesis of the Theorem implies that £xtp{£o,£o) is 
locally free for z > 3, so its Tors vanish (as do those of Homp{£o, £q) = O5). So 
the spectral sequence for £xtp{£o,£o (dp*J^) degenerates to £xtp{£o,£o) ^, and 
£xtp{£Q,£Q) is an obstruction sheaf as defined in ( 3.27| ). 



The necessary base-change for 7^^ follows from base change for £xtp. q.e.d. 



The condition required to employ the machinery of [LT|, [BF| to obtain a virtual 
cycle is then the following, as promised in (3.2). 



Definition 3.29 [LT| A4 has a perfect tangent-obstruction complex if there is a 
complex El — > E2 of locally free sheaves on A4 resolving the tangent obstruction 
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functors l \3. 26^ , 3.2^ ), in the following sense. Given standard data ( 3.24 ) we require 



that the cohomologies of the complex 

f*Ei®l ^ f*E2®T 
are T^^ (2) and T^^^ (g) Z, in degrees 1 and 2, respectively. Similarly for A4 l using the 

Remarks The first condition is easily seen to be equivalent to the exactness of 
E2 —>■ Ef ^ —5- 0, because of course the tangent functor assigns to data ( ^.24 ) 
the Os-module 

Tl{i) = noms{f*nM,i), 

by the standard deformation theory of the morphism /. One can also prove the 
equality Homs{f*O.M,I) = £xtg{f*£, f*£ (gil) directly using ( ^.261) to identify 



with £xtp ^{£,£ ^ Kx) (recovering the result of 0; here £^ is a local universal 

bundle on X x ^ as will be described below) and using some base-change 
and relative Serre duality | Ha3| ]. 



There is a weaker notion of perfect in (| |LT| ; comments following Corollary 3.6) 
which we will need later - namely that Ei need only exist locally on Ai, with E2 
still a global vector bundle on A4 surjecting onto T^. 



Theorem 3.30 Let X be a smooth, polarised, complex projective variety, and fix 
Chern classes ci G H'^^{X) and a line bundle L on X with ci{L) = c\. Let A4 denote 
the corresponding moduli space of stable sheaves, and A4 l the subscheme of those 



with determinant L { 3.2i ). If the numbers 

dim Ext* (£",£■), i>3, 

are constant over M 3 £ (e.g. if 'ExtQ{£,£) = y£ G Ai, Vi > then the tangent- 
obstruction complex of M given by Tl^, ^3_2di , \3.2$ ) is perfect. Similarly, for 



rank r > 0, the tangent-obstruction complex of given by 7^^^, {T^^)q is also perfect, 
as IS {TD^, {Tl\ for Ml. 



Remarks The required 2-step resolution is given in [pjT|| for X a surface, but the 
method does not generalise to higher dimensions. The proof below can be seen 
(see |ri[| ) as first working out the tangent-obstruction theory of the Quot scheme 
( p. 35 ) and then passing to the quotient M. of the relevant subset of Quot by the 



appropriate projective linear group ( 3.36| ) - but we shall work purely algebraically. 



We work in the universal case of S being (an open subset of) M. and resolve the 
sheaves £xtp{£,£ (gX), i = 1, 2, where I is an 0_A/(-module. Pulling back via maps 
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f : S ^ Ai gives all that we need; the only problem is that the universal sheaf £ 
on X X ^ may of course not exist. However we shall ignore this irritation 



since the usual methods (see e.g. [HL| 10.2) circumvent it - f exists locally on open 
subsets X X S {S C ^A) and choices differ by line bundles pulled up from S. But 
£xt*{£, £ ®I) is invariant under twisting by such line bundles and so exists uniquely 
and globally on X x A4. Thus all sequences in the proof below are really local, 
except those involving £xtpS, which patch together globally. 

Proof We denote by 0(1) the pull-back to X x 7W of the polarisation on X, by 
P{n) = xi^i'iT')) the Hilbert polynomial associated to the Chern classes q, and by 
£ the (local) universal bundle (see the Remarks above). Choose ni ^ such that 
£{ni) is generated by its fibrewise sections and has no other cohomology, i.e. we 
have a sequence 

O^J^^p*{p,£{ni)){-ni)^£^0, (3.31) 

for some kernel J^, and R!'p^{£{ni)) =0 Vi > 1. p^{£{ni)) is locally free as it has 
fibres of fixed dimension P(ni), and is flat over M because the other two terms 
are. 

Now twist with n2 sufhciently large such that J^{n2) and £{n2) are generated 
by fibrewise sections with no ^"^^ take cohomology: 

^p*(^(n2)) ^ p^£{ni) (^V ^ p^{£{n2)) 0. (3.32) 

V denotes p,,(0(ni — ^2)), and to pull the sheaf £{ni) through p^^p* we have used 
its local freeness. 

Define an 0^-flat sheaf % by 

^ OC ^ p*p^ je{n2) J(f{n2) 0. 

Then applying Homp{ ■ ,£{n2) ®p*T), for any 0_A4-module T, yields 

^ 7^omp(jr, £ (g) p*I) {p, Jf{n2)y p., £{n2) ® 1 (3.33) 

^ Homp{'X,£{n2) ®p*l) £xtl{Jf ,£ ® p*I) 0, 

where the final zero comes from the choice of n2 ^ 0, and the second term is 
produced by the projection formula for the 0;\4-flat sheaves p*p^:J^{n2) and £{n2). 
The higher terms in the above long exact sequence ( p. 331 ), with X trivial, give 

^ £xi^ {X,£{n2)) £xti+^ {Jt' , £) ^0, i>l, 
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while the long exact Homp{ • ,£) sequence of ( |3.31 ) yields 

^ SxtliJ^, £) Sxtp-^iS, £) ^0, j > 1, 
by the choice of ni S> 0. Thus 

£x-ep{%,£{n2)) ^ £xti+\jf,£) ^ £xt^+\£,£), 



(3.34) 



for all i > 1. 

The last term is locally free, by base-change and the constancy of dimExts in 
the hypothesis. Therefore so is the first term for i > 1, so its i = counterpart 

E2 ■.= nomp{X,£{n2)) 

is also locally free by base-change [ BPS| . Here we have used the facts that X 
is smooth and that the sheaves concerned are flat over A4, which also implies that 
Torj(f (n2),p*X) = for i > giving a spectral sequence ToTi{£xtp{JC, £{n2)),I) =^ 
£x£p~-' {%,£{n2) ®p*T) as in the proof of Theorem 3.2^ . By ( |3.34 ) this vanishes for 
i > 1, j > 1 since £xtj)^'^ {£ , £) is locally free for j > 1, so the spectral sequence 
degenerates to give 'Homp{X,£{n2) p*I) ^E2®T. Therefore (|3^ ) has b ecome 

^ nomp{J(f, £ Cd p*I) ^ E[(S)I ^ E2<S)I ^ £xtl{£, £ ® p*I) 0, (3.35) 

with E2 a vector bundle on ^A^, here we have defined E[ := (p* J^(n2))* ®p*f (712). 
Applying ?iomp{ ■ ,£ (^p*Z) to ( 3.31 ) yields 

7iomp{£,£ ® p*l) (p*<S(ni))* (8)p*<S(ni) (g) J 

^ nomp{J(f, £ Cd p*I) £xtl{£, £ (g) p*I) 0. 

The first two terms are I (by base-change since £ is stable, and so simple, on the 
fibres) and £ndMiP* £ {ni)) ® ^ with the identity map between them, giving 



— £ndQ{p^: £{ni)) —i- ?iomp{J^, £ (g p*I) 
Fit this into the sequence ( 3.35| ) 



i 



£xtl{£,£ (g)p*I) 0. 



£'no?o(p* £{ni)) (^2 

i \ 
nomp{.y(f,£ ®p*l) E[ 

i 

£xtl{£,£ (S)p*I) 

i 




>I ^ E2(g>I ^ £xtl{£, £ ® p*I) 
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(3.36) 



and divide by the two injections of £ndQ{p^: £ {rii)) to give the required sequence 

SxtliS, £ (g) p*I) ^ Ei(g)I ^ E2^I ^ SxtliS, S)(g)I ^0, (3.37) 

where I claim that Ei is locally free. To prove this it is enough to show the above 
map of vector bundles £ndo{p^ 8 (ni)) — > (p* J^(n2))* p*£{n2) = E[ is nowhere 
zero. But this follows easily by repeating all of the above analysis at a single point 
of M, instead of relative to M, and using base-change: the same exact sequences 
show that the map on fibres Endo(i?°(^(ni)) i/°(^(n2))* H^{8{n2)) is an 
injection. 

Finally, take the cokernel of the map i?^p*0 — > Ei, and/or the kernel of the map 
E2 — >■ R^p>fO, in the following diagram 

^ SxtliS, S (g) p*I) —^Ei(gl—^E2(gT—^ SxtliS, S ® p*I) 

id| |tr 

R^p^O^I R^p^Q®! 

t \ 

0. (3-38) 

For rank r > these maps are injective and surjective respectively, and so give 
locally free resolutions of the trace-free tangent-obstruction complexes, q.e.d. 



Corollary 3.39 Let X be a smooth projective 3-fold with trivial or anti- effective 
canonical bundle, let Ai denote the projective moduli space of semistable (with respect 
to the projective polarisation) sheaves of some fixed rank r and Chern classes, and 
Ml those sheaves of fixed determinant L. Suppose that all such sheaves are stable 
(e.g. if rank and degree are coprime). Then for r > there is a virtual cycle 
Zq C A4l, defined by the tangent- obstruction functors (T^)q, (T'^)q, of dimension 
the virtual dimension 



vd = "^{-iy+^dimExtl{S,S). 



i=0 



Its class in the Chow group A^diX) is independent of the resolution ( 3.3(\ ). For any 
r there are similar classes Z C M using the tangent-obstruction functors T^, (T^)o 
(of dimension vd + h^^^{X)) andT^,T'^ (dimension vd + h^'^{X) - h^^'^{X)). If M 
is smooth then the appropriate obstruction sheaf is locally free, and the virtual cycle 
is its top Chern class. 
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Proof Homo (£',£') = for any stable sheaf £ in the moduh space, so by Serre du- 
ahty ExtQ(f ,f) =Homo(^,^ ^ Kx)* and the assumptions on the canonical bundle 
Kx, Exto(f,f) vanishes also. Thus by Theorem 3.3C we may apply (0 3.7) to 
give the required virtual cycle, q.e.d. 

For r > we consider the trace- free obstruction class (T'^)q, since contains 
a trivial R^p^^O factor whose Chern class vanishes making the class of the virtual 
moduli cycle zero. For a Calabi-Yau manifold the above cycle Zq has dimension zero 
and its length will give our definition of the holomorphic Casson invariant. For this 
to be a sensible definition, however, we would like it to be deformation invariant. So 
we need to work out the deformation and obstruction theory of a sheaf £ on X as 
the complex structure of X varies over an affine curve. We will do this in the next 
section; meanwhile since has no complex deformations we can use the following 
Corollary to give an application of the virtual cycle. 

Corollary 3.40 Hilbert schemes of curves in a 3-fold X with trivial or negative 
canonical bundle (such as F^) have a virtual moduli cycle. 

Proof We use the particular moduli space of rank 1 sheaves of trivial determinant 
that contains the ideal sheaves of the curves. All we need to show is that any sheaf 
£ of the same Hilbert polynomial and determinant is also an ideal sheaf (these are 
then automatically stable). But since £, being stable, is torsion free, it is con- 
tained in its double dual, which is rank one, reflexive, and so a line bundle ([ PSS | 



pp 154-156). So it equals its own determinant 0, so C is a sheaf of ideals, q.e.d. 



Corollary ( 3.3S| ) allows us to define Donaldson-like invariants for such a 3- fold X 



by doing intersection theory on moduli spaces of sheaves using characteristic classes 
of universal sheaves, in the usual way. For instance we may now define (deformation 
invariant, by the results of the next section) Gromov-Witten-like invariants (with 
integral coefficients) using Corollary ( p.4Cl| ): choose n homology classes Oj in X of 
total codimension equal to the virtual dimension of a fixed Hilbert scheme H of 
curves in X, plus two. There is a universal curve C over H with a flat morphism to 
H, and similarly the projection of the nth fibre product to H, 

C xh ■ ■ ■ xh C —* H, 

is flat. Thus ( [p\i| 1.7) we may pull back the virtual cycle in i7 to a cycle Z in the 
total space and push it forward via the universal evaluation map 

ev : C ^ X inducing ef" : C Xh ■ ■ ■ Xh C ^ X", 
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to X" . Then the invariant is the integer intersection number (in the smooth variety 
X") 

e<(Z).(xr=ia,). (3.41) 

Similarly for a Calabi-Yau 3-fold X we may simply count the number of points in 
the virtual moduli cycle (i.e. take its length as a scheme) to get a count of curves 
in X. 

These invariants differ from the Gromov-Witten invariants since Hilbert schemes 
contain many nasty components representing things other than curves. An example 
that will also be relevant later (explained to me by Jun Li) is given by two disjoint 
s in 3-space coming together at a single point in a flat family. Consideration 
of the Euler characteristic of the structure sheaf, or just looking at the equations 
defining the subscheme, shows that the limiting curve must have a fat point at the 
intersection point (pointing in the direction in which the curves came together). In 
a separate flat family this point can break off to give a and a distinct point in 
the same Hilbert scheme. 

Remark A naive approach to creating a virtual fundamental class would simply be 
to take the top Chern class of the obstruction sheaf. This is correct if Ai is smooth 
and so we have an obstruction bundle. In general Pidstrigatch Q and Siebert |^ 
have shown the correct formula is the natural generalisation of this given (3.1, 3.2), 
namely the T;d-dimensional part of 

Z = c{Ei-Eo)n[cF{M)], 

where cf{M) is Fulton's total Chern class of the scheme M ( [pu| 4.2.6). 



Deformation invariance 

Fix a quasi-projective scheme X with a flat map to a smooth affine curve C, with 
projective flbres l : Xt ^ X over t ^ C. Given a stable (in particular, simple) sheaf 
£ on Xq we study deformations of its (stable and simple) pushforward to A", 
thus allowing it to move onto other flbres Xt (it is easy to see, using stability or 
simplicity, that this is all it can do: its support must remain over a flnite number of 
points in C and stability, which is an open property, prevents it from splitting over 
more than one flbre; the deformation theory below will show this for instance). 

We will need here and later a technical result, part of whose proof was worked 
out with the generous help of Brian Conrad. 
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Lemma 3.42 Suppose l : D C Z is a Cartier divisor in a quasi-projective scheme 
Z, with normal bundle v = 0£,[D). Then for coherent sheaves 8 and on D there 
is a long exact sequence 

Proof The sheaf sequence 0{—D) ^ — > i*0_D —>■ yields 

T-Com (0, L^T) nam (0(-L>), t^T) Sxt^i^Oo, i*^) ^ 0. 

The first map, multiphcation by the section of 0{D) defining D, is zero since is 
supported on Z). Thus we see that £'2;f^(t*0D, t*^) = v), so £xt^{i^E, i^^J^) = 

L^(7{om {E,T) for any locally free sheaf E on D. Thus, for a locally free sheaf 
E that is sufficiently negative (so that Ext-^ = H^{£xt^) in what follows) we have 

Exi^{i^E,i^T)^Yiom{E,J' ®v), (3.43) 

with higher Exts zero. 

Now take a (not necessarily finite) locally free resolution of i?* ^ £" ^ of £^ on 
D with the s (i = 0, 1, . . . ) sufficiently negative with respect to l^T as above (this 
is possible since D is projective and we may take E^^^ = H^{E^{N)) (S) Od{—N) 
for large N). Take also an injective resolution it^T ^ I' on Z (• = 0, 1, . . . ). 
Then the cohomology of the complex Hom*(t*£^, /*) computes Ext'{L^£, t^T). But 
/* is a complex of injectives bounded from below so respects quasi-isomorphisms 
like i^E' — > L^£ — > (since is exact). Thus the double complex Hom*(t*i?*, /*) 
also computes Ext*(i*f , i,,^). 

Now, by ( 3.43D above, the associated single complex Hom*(t*i?*, /*) (for fixed i, 



and with differential : Hom {i^E'^,P) — > Hom {l^E^, P~^^)) has cohomology only in 
degrees and 1. Thus, truncating all of these complexes (as i varies) simultaneously 
by setting terms in degree • > 2 to zero, and replacing the degree one term by ker 6j, 
we get a quasi-isomorphic complex B^'', sitting in an exact sequence of complexes 

^ ker 5° ^ B''' coker 6^ [-1] 0. 



(Here [—1] means shift the complex one place to the right.) So by ( 3.43| ) this sequence 
is just 

^ Rom{E\T) B''' Rom{E\T(S)i^) 0. (3.44) 

The above complexes compute ExV{£,J^), Ext*(i*f , i*.F) and Ext^~^ {£ , v) re- 
spectively, so taking the long exact sequence in cohomology of the exact sequence 
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of total complexes gives the required result, q.e.d. 



Remark The usual arguments show that the sequence is independent of choice 
of resolutions, and that the maps are the natural ones. The map Ext^(f,^) — > 
Ext^(t.*f , pushes forward by (which is exact) an extension of £^ by ^ on D 
to the corresponding extension of t*f by i^T on Z. The map Ext^(i*f , i*^) — > 
Hom£)(£^, i8) z^) is a little harder to describe - taking an extension class on Z and 
a (local) section of E on D, we must produce a section oi T ®v. But pulling back 
the extension (of i^E by l^T) by the section of E gives a section of Ext^ii^^D^ 
which is shown in the proof to be canonically isomorphic to l^,{!F ® v\ 



So in our usual set up ( |3.8| ) of 5 C y C Yi, consider the obstructions and 
deformations of a sheaf T := ti^E lya. X ^ Y , where if^ is a stable sheaf on Xq x Y 
(flat over Y\ and i denotes the inclusions C X and ^Y d X ^Y oi the fibre 
of X over E C. Restricting to 5 C y gives = i*'fo- 

The sequence ( 3.11 ) for the deformations oi T = i^E, and the corresponding 



sequence for E in one lower degree, are linked by the sequence of Lemma 3.42 (applied 
to the sheaves E and E ® J" ovl the Cartier divisor Xq x y c Af x y and written 
vertically below) to form the diagram: 



i m 



Ext^gxS'('^o,'£' 



n) 



■Ext^^^ 5(^0,^0®=^) 
Ext^xs(.Fo,-^o^^) 



i 



i 



1' 



Homxo X s (f , «) m) — ^ Hom^o x 5 (f'o , f «) n) — ^ Ext^^^, ^^{Eq.Eq® J). 

Here we have used the fact that the normal bundle to Xq x in x 5 is of course 
trivial. Since our extension class ejr G Ext^(.7i), JF n) is in the image of in the 
above diagram, the obstruction (9(ejr) = '-*c^o(e£') is in the kernel of (/>. Extending 
the right hand vertical sequence upwards therefore shows the following. 



Theorem 3.45 The obstruction map d of ( \3.1S ) takes values in coker((5), with 5 



the last map in the following sequence (3.4^ ) relating the first order deformations of 
To = L-*Eo to those of Eq: 

^ Ext^„x5(^o, Eo®y)^ Ext^x5(i*^o, ^ ^) (3.46) 

Homxox5(^^o, £"0 — > Ext^j,^ <j(£'o, E^ J). 

The first map pushes deformations on Xq x S forward to X x S. For Eq simple (e.g. 
stable) the penultimate term is just H^{J^), and 5 is the obstruction to first order 
deformations of Eq off the fibre Xq x S C X x S. 
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We want to repeat this result for trace-free determinants. Of course there can be 
an obstruction to deforming the determinant of a sheaf to a nearby fibre, so we need 
to assume this vanishes by fixing a determinant that extends to all of X. So choose 
line bundle L on X, and study stable sheaves on fibres Xt of X whose determinant 
on Xt is L\xf We will also now insist that rank£'o = r > 0. 



Then we showed above ( 3.45 ) that the obstruction to extending is the image 
in Ext^(.Fo,^o of the obstruction ^(e^:) G Ext^(£'o, (8) to extending £ 

from Xq X y to Xq x Yi. By Theorem ( p.23p this last obstruction in fact lies in the 
trace- free part ExtQ(£'o,£'o ® J^). So to get the trace-free analogue of the sequence 
( p. 45 ) we want to show that the map 



B.omxoxsi£o,£o ® — > Ext ^jjxs (^0,^0 ® ^) 
of ( 3.45| ) in fact factors through Extg. But this map fits into the diagram 



Homxox5(^'o,^o ® ^Ext^jjx5('^o,fo ® 

^iox5(^) Hl^.si^). 

where the bottom row is the corresponding sequence for detf, so the map 5 is the 
obstruction to extending detf, which vanishes by our assumption that deti5 is the 
restriction of a global line bundle L on X. Now this diagram commutes by the 
following observation of Brian Conrad, for which I am very grateful. Namely, the 

map Hom {£, Ext^(iS, T) of Lemma p.42| , is covariant in T and contravariant in 

£, making it covariant in Y{.T-Lom[£ , T) = Tiom' (E' , J^) , where E' is a finite locally 
free resolution of £. Therefore applying the trace map to Tni.om{£, £) gives the 
commutativity of the above diagram. 



So we can improve Theorem 3.45 to 



Theorem 3.47 Taking £q to have rank r > and determinant L\xq, the restriction 
of a global line bundle L on X , Theorem \3.4^ holds withExt'^^^g{£Q, £q(^J^) replaced 
by its trace-free part Extxgxsi^o^ ® ^)o- 



Corollary 3.48 Use the projections p : Xq x S ^ S and q : X x S ^ S , and let 

£ be a stable sheaf on Xq x Y (flat over Y ) with restriction £o to Xq x S such that 
dimExto(folxs) 'S'olxs) = for all s S and i >3. Then 

:= coker {O5 £xtl{£Q,£Q)} = image {£xtl{£Q,£Q) £xtl{TQ, J='q)} 
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is an obstruction sheaf for T = l^,E . If r > and det£'o is the restriction to Xq x S 
of a global line bundle L on X x S (pulled back from X), then 

C^o)o ■= coker {O5 fxtp(fo, fo)o} = image {fxtp(6'o, <So)o ^ £xtl{TQ,TQ)} 

is also an obstruction sheaf for T = i^8. Also, 

{T}Jq(I) := coker {iiVO ^ <SxtJ (^0,-^0 ^ 

(where the map is the identity into £xtp{£o,£o (E) J^) followed by the inclusion into 
Sxtg {Tq , J^Q ® J^) ) is a tangent functor for deformations of with fixed determinant 
L on the Xt fibres. 

Thus tangent- obstruction functors of the moduli problems for £ on Xq and T on 
X fit into the exact sequences of s -modules 

^ rlil) ^ T>^(J) ^l^Tl(S)I^Tl(E)I^O, (3.49) 

and 

^ {Tl),{I) ^ (T^Jo(^) - ^ - C?|,)o ® ^ - C^Do^^^ 0, (3.50) 
for any Os -module X. 



Proof Use the results of ( 3.28 , 3.26 ) for £q and = i*£'o, using the condition that 
the higher Ext groups of £q vanish to show the same for (this follows from the 
exact sequence ( |3.42| ) as usual.) Then Theorems 3.45 , 3.47| give the required exact 
sequences, q.e.d. 



Theorem 3.51 The tangent-obstruction functors of £ on Xq (\3.2(\ , 3. 28^ and of 



J- = i^£ on X ( 3-4^ ) are compatible in the sense of Li-Tian ( %J^ Definition 3. 



Proof The compatibility of Li-Tian says roughly that for an obstructed sheaf £ 
on Xq to extend without obstruction inside X, the obstruction must cancel the 
obstruction to extending i^.<f^o in the direction of the base C. 

The precise statement is that there should be an exact sequence ( p9|) (or ( ^!50|) 



in the trace-free case) such that the following holds. Let 5 C y C Yi be as in 
(p^), and take a sheaf £ on XqX Y, giving a corresponding class e € Ext^{£Q, £ i^n). 
Suppose that the obstruction class (?o(e) G Ext^(<f^O) '^o'X' J^) to extending it to XqxYi, 
has vanishing image in Ext^(.Fo,^o "X" J^). Thus = i^{£) extends to a sheaf on 
X xYi, giving a class / G Ext^(^O) ® Tfi) and a diagram 
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e G ExtHf^o,^' ® n) 
/ G Ext^ {To, J" m) ^ Ext^ (JS, -5^ ® n) 
^ Horn {£q, £q0 J^) ^ Horn {£o,£ m) ^ Horn (fo, ^ n) 

I' 

do 



ee Ext^(£'o,£'(8)n)^Ext2(£'o,£'oO=y), 



where all horizontal maps come from sequences of the form ( |3.11| ) , and vertical maps 
from ( |3.42| ). Since i*(e) is mapped to zero under ip, 4>{f) is in the image of a unique 
class c G Hom (£^0; ® '-^)- Then Definition 3.8 of |LT| requires that 5{c) = —do{e). 

In our situation this holds by abstract homological algebra. We first need to 
lift the big commutative diagram of Ext groups that the above diagram is a part of 
to the level of short exact sequences of complexes. Given an O^-module J, we use 
the complex By, whose cohomology is Fixt'(J^Q,J^ J), from the proof of Lemma 
3.42. This is functorial in J, and we denote the other two complexes in ( 3.44]) by 
A J and Cj (computing the cohomology of Ext* {£o ,£ ® J) and Ext*~^ (<fo ,£ J) 
respectively) . 

Thus we get a (horizontal) sequence of sequences of the form ( p.44| ) (written 

vertically) by setting J to be the different modules in the exact sequence Q ^ ^ 
m ^ n — > 0, yielding the exact commutative diagram of complexes 



1 . 1 



4* 1^ A' Jl^ A' 



jt.^ jtm jin 

|p.# |pm |pn 

C* TTC 
J ^ '-^m ^ '-"n 



1 




III 
0. 

We now follow the previous diagram chase around this diagram at the level of com- 
plexes. So we start with e G with (5e = (we will denote all coboundary operators 
by 5) which we want to lift to / G B\^ with bf = 0. 

Lift e to /3 G A}^ (using the fact that ■ka is onto). This is then assumed not 
coclosed; its coboundary is by definition (^(/3) = jA(9o(e)) where 5o(e) G rep- 
resents the obstruction class 9o(e) G Ext^(f^O) -^o ® -^) to lifting e. Pushing forward 
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/ := i^miP) to a class in B^, we see that a coclosed class / € i?^ lifting tn(e) € 
exists if and only if there is an a G i?^ with (5(i_B(a)) = —6{f); the required / is 
then f = f + jsict). We are assuming /, and so a, exist. 

Thus, since (5(a)) = -6{f) = -6{Lm{P)) = -i^mjA{do{e)) = -js (9o(e)), 
and since Jb is an injection, we have 

S{a) = -L,y{do{e)). (3.52) 

In particular then, p^f{a) is coclosed, defining a class c GHom(£^0)^o ^ '-^) whose 
coboundary (by which we mean now the connecting homomorphism 6 in the coho- 
mology sequence of the left hand column that arises in the previous diagram) 5{c) 
is -do{e) by (PI). 



Thus we are left with showing that c is the same c as defined above, i.e. that 
jc{c) is Pmif)- But jc{c) = jcpy{a) = PmjB{a) = Pmif - f) = Pmif) as required, 
since / = LmiP) is in the kernel of pm- q.e.d. 



Corollary 3.53 The virtual moduli cycles of Corollary 3.3^ are deformation in- 
variant in the following sense. Given a family X ^ C of smooth projective varieties 



Xf, t ^ C , consider the family of moduli spaces Ait of stable sheaves £ as in (3.3§ ) 
on the fibre Xt (containing the virtual moduli cycle Zf). These form the moduli space 
Ai ^ C of sheaves {it)*£- Then under the same conditions as Corollary 3.3% M 
has a virtual moduli cycle Z of dimension dim Zt + 1, and as elements of the Chow 
group they satisfy i\Z = Zt- (Here i\ is the Cysin homomorphism 6.2) of the 

inclusion it : {t} C.) There is also the corresponding result for sheaves of fixed 
determinant Lf, with L a fixed line bundle on all of X , for rank r > 0. 

Proof We first need to show that the tangent-obstruction functors of ^ C are 
perfect in the weaker sense of the Remark preceding Theorem |3.3C . The E2 of that 
Theorem exists on all of M and surjects onto 7^^, and this in turn surjects onto 
T^^- So all we need is a local vector bundle Ei giving a resolution of T^^^, 7^^^ over 
an open set of M.. 
The map 

Oa^ Tl, of (|3l9| ) locally factors through 
^ E2. Combining this lift with the map Ei — s- E2 of Theorem |3.30| into a map 
El Q —I- E2 gives an exact sequence, for any 05-module I, 



via a local lift 



^ TUz) ^ {Ei®0)®l ^ E2®1 



T2 



0, 



by combining ( p. 37 ) with ( |3.49| ). Thus we can use [LT] to produce the virtual moduli 
cycle Z (and similarly for the trace- free versions, using (|01 ) and (^) instead). 
ilZ = Zt follows from ( jLT] J 3.9), given the compatibility of Theorem |3. 5 1[ q.e.d. 
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The holomorphic Casson invariant 



Definition 3.54 Fix a smooth projective Calabi-Yau 3-fold X, and a rank r and 
Chern classes Ci such that the moduli space Ai of semistable sheaves with this data 
( and fixed determinant L if rank > j contains only stable sheaves (for instance if 
the rank and degree are coprime). Then we define the holomorphic Casson invariant 
\^c.j{X) to be the length as a scheme of the zero dimensional projective virtual 



moduli cycle Zq C Ml of Corollary 3. 3i\. It is invariant under deformations of X 



in any projective family to which L extends (e.g. if h^''^(X) = this is immediate). 
The deformation invariance comes from the relation l\Z = Zt of ( 3.53| ) and the 



resulting "conservation of number" (|Fu| 10.2). This invariant is clearly similar in 



nature to the Gromov-Witten invariants of X. In fact we might expect to recover 
GW invariants either by considering moduli of ideal sheaves of curves as in ( |3.40| , 



3.41 ), or by relating rank two bundles to curves via zero sets of their sections (and 
vice- versa by the Serre construction). However, we have already remarked that 
the first case gives something slightly different to GW invariants. As for rank two 
bundles, under the Serre construction spheres and tori tend to correspond to unstable 
bundles, and for a higher genus curve to correspond to a bundle its tangent bundle 
must extend to a line bundle on X, cutting down the space of admissible curves. 
Since the space of curves has expected dimension zero anyway we tend to find (for 
instance in the examples below) that rank two bundles correspond to non-generic 
high-dimensional families of curves. In this case deforming the curve corresponds to 
deforming the section, not the bundle, so GW invariants do not arise. 

We might also like to count fewer singular sheaves, i.e. to only count bundles. We 
would expect from Donaldson theory to have to include sheaves with codimension 
two singularities (limits of stable bundles, where "bubbling" occurs), but in the 
rank two case one might hope to be able to ignore sheaves with codimension three 
singularities. However again the example above of a flat family of curves producing 
a distinct point shows that bundles can degenerate to sheaves with codimension 
three singularities, and so such sheaves can lie in the same connected component as 
bundles. 

In all of the examples we consider, however, we will be able to show there are no 
such singular sheaves, and in fact GW invariants will arise in a completely different 
way in the last section. 



Examples Simple examples of the invariant are given by considering ideal sheaves 
of 1, 2 or 3 points in a Calabi-Yau 3-fold X. The moduli space is then X, Hilb^X 
or Hilb^X, and so smooth, with the invariant giving the Euler number of the cotan- 
gent bundle (since this is the obstruction bundle), i.e. —xiX), x(Hilb^X), and 
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— x(Hilb^X) respectively. 



A deeper example is motivated by Donaldson's reinterpretation |DT| of work of 



Mukai ( pull] 0.9) as an example of the Tyurin-style Casson invariant of ( ^.T]) . 



Fix a smooth quadric Qo in P^, in a fixed P^-family of quadrics spanned by 
Qo, Qi and Q2, say. The singular quadrics in the family lie on the sextic curve 

C = {[Ao; Ai; A2] G : det(Aogo + AiQi + X2Q2) = 0} C P^ 

where the quadratic form defining the quadric becomes singular. 

For every point of P^ \ C we get two tautological rank 2 bundles A and B over 
the corresponding quadric (thinking of it as a Grassmannian of 2-planes in C^, A 
and B are defined by the tautological sequence 0^^* — >C^— >i?^0), and so 
also over the K3 surface 

5 = QonQinQ2- 

In fact these A and B bundles give a double cover Ai of P^ branched along the sextic 
curve C (the A and B bundles coincide on the singular quadrics) as the moduli space 
of bundles of the same topological type over S. is a (complex symplectic) K3 
surface, notice. 

Similarly the Fano Xi = QqCi Qi lies in the pencil spanned by Qo and Qi, a line 
P^ in our P^-family. The 2-fold branched cover of this line induced hy A4 — > P^, i.e. 
the set of A and B bundles on the quadrics in this pencil, is the moduli space for 
Xi. Similarly for X2 = Qo Ci Q2 and the cover of the line {Xo, X2) C P^. 

So we have an example of the Tyurin-Casson invariant ( |2.7| ), with two complex 
Lagrangians (the curves covering the P^s) as the moduli spaces of bundles on the 
two Fanos, injecting into the complex symplectic moduli space of bundles on the 
common anticanonical divisor S. Their intersection, namely the double cover of the 
intersection point {Qo} of the lines in P^, corresponds to the two stable bundles Aq^ 
and Bq^ on the singular Calabi-Yau that is the union of Xi and X2- 

Deforming this singular quartic in Qo to a smooth Calabi-Yau we would like, 
then, to prove the following. 

Theorem 3.55 Let Qo be a smooth quadric in P^, and let X be a smooth quartic 
hypersurface in Qo- Then the bundles A and B on Qo restrict to stable, isolated 
bundles of the same topological type on X, and they are the only semistable sheaves 
in the moduli space. Thus the corresponding holomorphic Casson invariant is 2. 

Proof Standard exact sequences and geometry on the Grassmannian Gr (2, 4) = Qo 
show that the bundles are stable and isolated ([Tl] 2.3.1). The more difficult part 



is to show that any semistable sheaf £ of the same Chern classes is either A B. 
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We will do this by controlling if^'s cohomology by studying it on hyperplane sections, 
and then using Riemann-Roch to produce sections (c.f. | Ha2(| ). 



Let £ denote the double dual (or "reflexive hull") of £". Let F = F"^ n X be a 
smooth hyperplane section, with C sufficiently generic that £\h '^s the double 
dual oi S\h and so a bundle F, say. Then the Riemann-Roch formula for F is 

2/i°(F) - h^{F) = 12 - C2(F), (3.56) 

by Serre duality, Kh = 0_f/(l), and an unpleasant computation. Thus F has at least 
4 sections, as C2{F) = C2[£) - to < C2{£) .w = 4 (recall that passing to double duals 
lowers C2 because of the exact sequence embedding a sheaf inside its double dual). 

Also, as H is generic, we may assume its only line bundles are the 0(n) bundles, 
by Noether-Lefschetz theory (see e.g. [ |GH2 |). £ is slope semistable and so slope 



semistable on restriction to the generic hyperplane (see e.g. [ Ha2 | 3.2). Thus F{—1) 
cannot have any sections, so the four sections of F must vanish only on points, giving 
us a Koszul resolution ( [|GH | p 688) of the form 



O^O ^ F ^ ^c(l) ^ 0, (3.57) 

where J^c is the ideal sheaf of functions vanishing at c = C2{F) < 4 points. Taking 
sections and using the Riemann-Roch formula ( p.56| ) gives 

h\y,{l)) = h\F) - 1 > 5 - c/2. 

However /i''(Oi^(l)) = 5, and the c points impose at least min(2,c) conditions on 
the sections of 0_f/(l) as they are the restriction of the sections of 0p4(l) on the P^ 
hyperplane in P^ Thus 5 - c/2 < /i°(j^c(l)) < 5-min(2,c), i.e. c > 2min(2,c), 
whose only integral solutions for < c < 4 are c = and c = 4. We can rule out 
c = by stability (either by the Bogomolov inequality or the fact that ( |3.57| ) would 
split); thus c = C2{F) = 4. Therefore £ has the same second Chern class as £ on X 
(since H'^{X; C ) is generated by a class non-zero on H), and can only differ from it 
in codimension three. 

Notice also that as we must have h^{^4{l)) = 3, the points lie on a web of 
hyperplanes in the P"^ hyperplane, i.e. on a line in P^. 

To control the cohomology of F we use ( 3.57] ) to give, for t > 2, 



0^ HHF{t-l))^ H\.Mt))^ H\OH{t-l)) ^0, 

H^{F{l-t))* H^{0H{2-t))* 

by stability. But the sequence =^4(i) Onit) 04{t) shows that 
h^{^4{t)) = Vt > 3 since we can find a polynomial of any degree > 3 taking any 
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prescribed values at 4 points on a line (in P^). Similarly /ii(j^4(2)) = 1, and we get 
i?i(F(-n)) = Vn > 1. 

Thus we can now pass up to X using the sequence {) ^ £ {—1) ^ £ ^ £\h ^ 
which is exact because £ is torsion free, giving 

H\£{-n - 1)) ^ H\£{-n)) ^ H\F{-n)). 

Since H^{£ (— vanishes for large n it therefore vanishes for all n > 1; in particular 
H'^{£ ) = 0. This and stability simplify the Riemann-Roch formula for £ to 

h\£)-h\£) = A + cS)l'^- 
The third Chern class of a rank two reflexive sheaf on a smooth 3-fold is always 



nonnegative and vanishes if and only if the sheaf is locally free (| Ha2 ] 2.6), so £ has 
at least 4 sections which do not vanish on divisors, by stability. Thus we have a 
presentation ( [|Ha2| 4.1) 

^ ^ f ^ J^t(l) 0, 

for some degree four curve C . Computing the third Chern class of such an exten- 
sion to be zero shows that £ is locally free with the same Chern classes as £, i.e. 
£" = £" is locally free. (The point here is that the extension is locally free, and not 
just reflexive, because the determinant of £ , restricted to C, is isomorphic to the 
determinant of the normal bundle to C, with the isomorphism set up by the deter- 
minant of the section. Thus the section vanishes transversally along C, the above 
sequence becomes the Koszul resolution ( |3.57| ) for this section, and £ is locally free. 
In fact the Serre construction ( ]OSS | p 93) uniquely constructs a sheaf £ (which 



is then locally free) from the above resolution when the appropriate determinants 
are equal; it is only when they differ that reflexive sheaves aride from Hartshorne's 
generalisation of the Serre construction | Ha2f| .) 



But now h^{^c{^)) > 3, so C lies in a web of hyperplanes m X d Qq C P^. 
Thus it lies on a linear P^ plane P C P^. Since C lies in the quadric Qq, the plane 
P must do too, otherwise the quadric would intersect P in a conic curve containing 
the degree four curve C, which is impossible. 

But the planes in Qq are precisely the standard planes in Gr (2, 4) - zero sets of 
sections of A and B. P uniquely defines either the A or the B bundle on Qq via the 
Serre construction (see e.g. lOSS p 93) 



^ ^ A/B J^p(l) ^ 0, 

by the extension data Ext"^(j^p(l), Oqo) = H^{Op) B 1. This extension restricts, on 
the quartic X, to 1 G H'^{Oc) = Ext^(j^c(l), Ox), defining our bundle £ 

0^O^£ ^ J^c(l) ^ 
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by uniqueness, since H^{Oc) — C (C is a curve in P^, so is connected). Therefore 
£ is one of ^ or S restricted to X. q.e.d. 



4 K3 fibrations 

We now turn to calculating the invariants on iCS-fibred Calabi-Yau manifolds with 
no reducible or multiple fibres, using the nice properties of moduli of bundles and 
sheaves on a K3 due to Mukai ( [ [HL| Chapter 6) . These do not quite generalise to the 
singular K3 fibres. Although these have a (trivial) dualising sheaf as the fibres are 
complete intersections ([Hal| III 7.11) and so the usual Serre duality holds ( pal|| 



III 7.6), the Serre duality we have been using [ Mul| , 



Ext*(^,^)* ^ Ext^-^^,^), (4.1) 

only holds for sheaves £ with a finite locally free resolution. On the singular fibres 
there will be sheaves with unbounded locally free resolutions for which the result 
does not hold. 

We will consider stability on X using a suitable polarisation lo, in the sense of 
Friedman. This means that the fibres are small so that semistable sheaves restrict, 
on the generic fibre, to semistable sheaves. The basic idea is to add a large multiple 
of the fibre divisor / to any fixed polarisation, so that the sign of the degree uj .uj .ci 
of a possibly destabilizing subsheaf is the same as the sign of the degree /. u; . ci on a 
generic fibre. Since such subsheaves that we need to consider form a bounded family 
we can do this: 

Proposition 4.2 Let X be a surface-fibred projective 3-fold. Choose rank and 
Chern classes such that slope semistability implies slope stability on smooth fibres 
(e.g. if rank and degree are coprime). Then we may add a sufficient number of fibre 
classes to the polarisation such that sheaves of the given rank and Chern classes are 
stable if and only if they are stable on the generic fibre, and there are no strictly 
semistable sheaves. Such a polarisation is called suitable. 

Proof This should really be proved directly, but a cheat goes as follows. Choose 
^ such that any slope semistable sheaf £ (of the given Chern classes) restricts 
to a slope semistable sheaf on a generic hyperplane H in the linear system |0(A)| 
([[HL[ 7.2.1). Now there is an M such that (w + M f )\H is a suitable polarisation on 



iif — > C by (|HL| 5.3) (here uj denotes the Kahler form on X, and / the class of a 
fibre). Thus £ is slope semistable on the generic fibre Ht of H ^ C, and so slope 
semistable on the generic fibre of X — > C (we are using the easy fact that the 
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(slope/semi) stability of a sheaf on a hyperplane section implies the same on the 
whole space). By assumption, then, £ is slope stable on Xt. 

Reversing the argument, by increasing if necessary, if £ is slope stable on Xt, 
then by a theorem of Bogomolov ([HL| 7.3.5) £ is stable on Ht- Since {oj + M f)\H is 
a suitable polarisation, this implies that £ is slope stable on H, and so on X. q.e.d. 



Definition 4.3 Let X he a smooth polarised 3-fold, K3-fibred over a smooth curve 
C, let Ci € H^'^{Xt), i = 1,2 be Chern classes in the cohomology of a generic fibre, 
and let r € Z>o be a rank. Then we say that {X, r, Cj) is admissible if and only if 

• X ^ C has no reducible or non-reduced fibres. 



the polarisation on X is chosen to be suitable for {r,Ci) using Proposition J^.i, 



• ci is the restriction of the first Chern class of a global line bundle on X , i.e. 
is in the image of H^^{X) —>■ H^^{Xt), 

• gcd{r, c\ . Lo, — C2) = 1 (where uo is the Kdhler form of the induced polari- 
sation on the fibre), and 

• on any fibre Xt, slope semistability of sheaves with Chern classes {r,Ci) implies 
slope stability (e.g. if rank r and degree ci .u;|xt are coprime). 

We then set d = 2rc2 — (r — l)cf — 2(r^ — 1), the dimension dimExt"'^(f , (?) = 
l)*"*"-^ dimExtQ(f , (?) (by Serre duality and stability of £) of any nonempty 
moduli space of stable sheaves £ of Chern classes (r, q) on a smooth fibre, d is 
even. 

The penultimate condition ensures that a universal sheaf exists on the product 
of any fibre and its moduli space ( |lHL|| 4.6.6), and also that semistable sheaves on 
the fibres are in fact stable ( [|HL| 4.6.8 - here is where we use the assumption that 
the fibres are reduced and irreducible to ensure the ranks of possible destabilising 
subsheaves are integers.) Thus the fibre moduli spaces are fine. But in general 
this may not be enough to ensure slope stability, so we require the last condition. 
Under these conditions we will often just talk about stability. For some results we 
will restrict to fibrations X ^ C whose singular fibres have only ordinary /rational 
double point (ODP/RDP) singularities, since we then understand something about 
reflexive sheaves on such i^3 fibres (e.g. |^], |pa|]). 



Definition 4.4 We say the above data (^j_S) is very admissible if each singular fibre 



of X contains only a single GDP, and r and ci . u are coprime. 
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Theorem 4.5 Choose an admissible triple (X, r, Cj) as in Definition Then 
there is a projective scheme A4 ^ C with fibres A4t that are the moduli spaces of 
stable sheaves of Chern classes {r,Ci) on Xf. Suppose that for some smooth fibre 
Xt, A4t / 0- Then A4 ^ C is surjective with generic fibres smooth of dimension d, 
and there is a universal sheaf T on X Xc M. (unique up to twisting by the pull-back 
of a line bundle on C). 

Suppose now that the singular fibres of X ^ C have only RDPs. If d = 2 and 
r > 2 then all elements of Ai are reflexive on their supporting fibre (and so locally 
free on smooth fibres). If d = and r > 1, T is locally free, each Ait is a single 
reduced point, Ai = C, and so T is a bundle on X . 



Proof Simpson's construction of a projective proper Ai ^ C is now standard pIL(| , 
given the assumption that there are no semistable sheaves. If we have a stable sheaf 
£ in the moduli space of a smooth fibre Xt then its deformations are unobstructed 
since by Serre duality ExtQ(<?,<?) = Endo {£)* = and detE' is unobstructed by the 
assumptions on ci. Thus it may deformed off Xt (|3.45| ), making Ai ^ C onto over 
an open set of C, and so onto all of C by properness. Since Extg(iS,iS) = 0, Ait is 
smooth of the correct dimension d. 

Consider sheaves £t S Ait to be torsion sheaves on X, by pushing them forward 
to {it)9f£t, where it : Xt ^ X \s the inclusion. Then Ai ^ C \s part of the moduli 
space of sheaves on X of the same Hilbert polynomial (in fact deformation theory 
and stability show it an entire component of the moduli space). There is a universal 
sheaf on X X as the numerical conditions of ( pIL| 4.6.6) are satisfied by {it)t:£t'- 
they are satisfied by £t on Xt, as gcd (r, ci . lo, ^cf — C2) = 1, and they only depend 
on the Hilbert polynomial, which is the same for £t and {it)i(£t- 

The universal sheaf on X x is supported on the image of the diagonal map 
X Xc M ^ X X Ai, and so defines T on X Xc M. 

The statements about reflexivity of sheaves of rank r > 1 are standard arguments 
of Mukai ( piL| 6.1.6, 6.1.9) for smooth fibres - taking the double dual of a sheaf 
does not affect ci but decreases C2 (look at the exact sequence of the sheaf injecting 
its double dual, which it does if r > 1 since it must then be torsion- free by stability), 
which decreases d (|4.3| ) by 2r times as much. Since the double dual is also stable it 
sits inside a moduli space of the correct dimension, thus d{£ ) must be greater than 
or equal to zero. Thus if d = 0, or d = 2 and r > 2, the sheaf must be its own double 
dual and so refiexive. 

For a sheaf f on a singular fibre Xt with only RDPs we again take its double 
dual £ , and then pull this up to the minimal desingularisation vr : Xt — > X^ of 
the fibre (another K3), and divide by torsion. This gives a vector bundle £ on Xt 
which we show is stable with respect to the (degenerate) polarisation 0(1) pulled 
up from Xt. Any subsheaf T ^ £ can be pushed down to a subsheaf of vr^Kf , and 
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■K^E'^E, R^TT^S = (see e.g. Q). So for n » 0, 

xiJ'in)) = x{^*Hn)) - xiR'TT.T) < xi^^Hn)) < x{£ (n)) = x{^J{n)) = x{£{n)) 

by the stability of £ , demonstrating the stabihty of £. Thus it is also stable for 
nearby nondegenerate Kahler forms. 

In particular £ is simple, so that its topological invariant d{£) ( [4. 3D gives the 
dimension Ext^ (£",£") of the moduli space it sits in. This must be greater than or 
equal to zero, so the previous argument (in the smooth case) goes through as before 
to show that £ is locally free for d{£) = 0, r > 1 and reflexive for d{£) = 2, r > 2, 
if we can show the inequality 

d{£) <d{£), (4.6) 

with equality if and only if £ is locally free (notice that we already know that 
d{£ ) < d{£) with equality if and only if £ is reflexive). 

So we want to compare the topological invariant d on smooth fibres with d{£ ) 
on the resolution of the singular fibre Xt- This is more-or-less contained in the 



work of Langer |La], and I am grateful to him for explaining it to me. He defines a 
second Chern class for reflexive sheaves such as £ on singular surfaces such as Xt, 
which we will denote by C2 (^ ) . This is not the same as what we will denote by C2 , 
namely the class which gives the right contribution to the Riemann-Roch formula 
(the deformation invariant C2{£) := C2{£) — I, where I is the length of the torsion 
sheaf cokenel of £ ^ £ , and C2{£) is measured on a nearby smooth fibre). 

Langer's definition (building on work of Wahl) is given in terms of the sheaf 
£ = {tt*£ I torsion) upstairs, as ( [pa|| Section 3) 



c\(£):=c2{£)-Y,c2{£,y), 



where y runs over the RDPs of Xt, and Ci{£,y) is the local ith. Chern class on 
its resolution (|La] 2.2, 2.3). He defines ( pla| 2.7) ay{£) to be the local difference 



between C2 and the C2 we use that fits into the Riemann-Roch formula; that is 

C2{£)=C^{£)-Y,^y{£). 

y 

The last result of (|La| Section 6) is that for an RDP y, 

= \(-i{£^y) - C2{£,y), 
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giving 

y 

Also, the local first Chern class satisfies 

cl{£)-cl{£) = Y,cl{e,y), 

y 

so that putting these two formulae together gives 

[2rc2 - (r - l)cl - 2{r^ - 1)] {£) - [2rc2 - (r - l)c? - 2{r'^ - 1)] {£) 

= Y,cli£,y). (4.7) 

y 

Thus d{£) — d{£) = X^ycf(£',y) is the sum of squares of divisors like —ci (£",?/), a 
positive multiple of which is effective and supported entirely on the exceptional set. 
This is therefore negative, and zero if and only if ci (£",?/) = for all y, if and only 
if £ is locally free at all singular points y Q. 

Finally, we want to show that if d = then Mt is a single point. This is an 
argument of Mukai for smooth fibres ( piL(| 6.1.6) which generalises to singular fibres 
with RDPs since we have just shown that the sheaves are locally free, so the duality 
Ext^(f,£:)* ^Ext^-* (£■,£■) gll) holds, which is all that is used, q.e.d. 



Alternatively, in the d = case, we can give an easier proof of the existence of T: 
using the same diagonal map X Xc -M. ^ X x M and Luna's etale slice theorem for 
the quotient map from the relevant Quot scheme to A^, T exists locally over C. It is 
simple, so is patched together by nonzero scalars on overlaps of open sets. Thus the 
obstruction to the patchings satisfying the cocycle condition lies in ff^(O^), which 
is zero. We start by analysing the holomorphic Casson invariant in this d = Q case. 



Theorem 4.8 Let (X, r, Cj) be as in Definition ^Tj, K3-fibred over C = P , with 
d = 0, r > 1. // the fibres have only RDPs then the bundle T ^ X constructed in 
Theorem \4.d^ is slope stable, isolated, and the only semistable sheaf with the same 
Chern classes. 



Remarks Though the method of this proof and that of Theorem 4.18 below could be 
applied to other such "adiabatic limit" problems for stable bundles in any dimension, 
the conditions we require (that the fibre moduli spaces should be empty for stable 
sheaves of the same rank and ci, lower C2, and any Ci i > 3) are so stringent that they 
are probably only effective on K3- (or, with some modifications to take account of 
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the fundamental group, T^-) fibred 3-folds, and Fano-surface-fibred 3-folds. Again, 
allowing for the fundamental group (i.e. fixing determinants), we can extend the 
above to any base curve C, but we restrict to for simplicity and because it is the 
case relevant to Calabi-Yau 3-folds. 

Notice we are not claiming that the moduli spaces are so simple for any classes 
on the total space satisfying 2rc2 — (r — l)cf — 2(r^ — 1) = on the fibres. Even when 
a stable bundle with these classes exists on a fibre, we could modify T by giving it 
ideal-sheaf singularities contained in fibres, or make it unstable on some fibres, or 
add codimension 3 singularities, to produce stable sheaves in different, more com- 
plicated moduli spaces. What is remarkable about T's moduli space is that no such 
singular phenomena can occur. 



Corollary 4.9 Any such K3-fibred 3-fold admits isolated bundles if the generic fibre 
does (with ci a class coming from the total space), and with respect to some polari- 
sation these bundles are stable and the only point in their moduli space of sheaves. 
In particular, the relevant holomorphic Casson invariants of K 3- fibred Calabi-Yau 
3-folds are one. The same is true for the dual Chern classes (— l)*Cj G H^''^{Xt). 

Proof The point here is to show that the Casson invariant is defined, i.e. that 



semistability implies stability. But this follows from the choice of polarisation (4.2), 
implying that slope semistability implies slope stability, which is stronger. 

As for deformations, in a polarised family of Calabi-Yau manifolds the fibration 
structure survives (the obstructions to the survival of a generic fibre Xt in a defor- 
mation lie in H^{vxt\x) — H^i^Ks) = 0), and the suitability of the polarisation is 
preserved (the volume of the fibres is unchanged, for instance). 

For the dual Chern classes we take the dual bundle on a smooth fibre and repeat 
the construction, giving the unique bundle T* in the moduli space, q.e.d. 

Proof of Theorem Denoting the projection map by tt : X ^ F^, 1 claim that 
Ext\{T,T) is zero by standard base-change arguments. Namely: pick a finite, very 
negative, locally free resolution E' T, so that Ext^(E^XtiT\xt) = for i > and 
t G P^. Then Sxt'^iT, T) is the cohomology of the complex F' = p^ 'Hom{E' ,T). T is 
flat over P-*^ so we can restrict to any fibre Xt to give a resolution oiT\xt'-, thus F'\xt = 
'Rom.Xt{E'\xt,T\xt) computes Ext3f {T\xf,T\xt)- Since the higher Exts vanish by 
construction, and since T is flat over P^, the dimension of each HomXi(-£'*|xt,7"|xt) 
is constant in t. Thus F' is a complex of locally frees. By stability, its zeroth 
cohomology on each fibre is canonically End(T|xf) = C . id, so we get a nowhere 

vanishing map Opi ^ ker (FO U F^), and the image d^{F^ /Of,i) C F^ is locally 
free. Therefore £xt\{T,T) is the kernel of the map d^ that d^ induces on the locally 
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free cokernel of dP. Similarly Ext^ {T\xt,T\xt) is the kernel of d^\xt on cokerd°|xf 
But this is zero for Xt a smooth fibre, so that is generically an injection of vector 
bundles. Thus its kernel, as a map of sheaves, is zero, and Sxt]^{T,T) vanishes. 

So the Leray spectral sequence for 'K^£ndQ{T) and £xt]^{T,T) shows that T is 
isolated. (Alternatively, the proof below extended to X x Spec C [e]/ (e^) shows there 
is a unique bundle on this thickened space.) Stability follows from fibrewise stability 
and the choice of polarisation. 

What we want to prove is that any stable sheaf £ of the same Chern classes is 
isomorphic to T. The basic idea is that, firstly, taking double duals decreases C2 
and the fibrewise moduli spaces of lower C2 are empty, and secondly, that if sheaves 
have fibres on which they are unstable, this only ever increases C2. Playing these 
two phenomena off against each other ensures we have no non-reflexive sheaves, and 
no unstable fibres. 

We first replace the stable sheaf £ by its double dual £ , which on the generic 
fibre is the double dual of the restriction of £ to that fibre, and hence slope stable 
and locally free there. 

We have a sequence 

0^£^£^J^^0, 

where is a sheaf supported on a subvariety Z of codimension two or higher. 
Therefore 

C2iT).io = C2i£).io = C2i£).uj + J rkziJ^)u;>C2i£).io, (4.10) 

with a similar inequality C2(£\xt) < C2{T\xt) on a generic fibre Xt. But the dimen- 
sion d of the moduli space on a smooth fibre decreases by 2r > 2 for every decrease 
in C2, and must remain non-negative on the generic fibre where £ \xt is stable. Since 
ci = we see that £ \xt is reflexive and in the same moduli space as T\xt, and so it 
is T\xt, recalling that the moduli space is a single point. 

Therefore, using stability, on the generic fibre Hom {T\xt,£ \xt) is a copy of C , 

so 

TT* Horn (T, £ ) 

is a rank one torsion free sheaf on P^, i.e. 0(— n) for some n. This gives the exact 
sequence 

^ r(-n) ^£ ^0, (4.11) 

where T{—n) denotes the twist of T with the pullback of 0(— n) from P^, and Q 
is a sheaf supported on some finite number d of possibly singular fibres {X^. j^^^^. 
(Some of these fibres might be infinitely close; d is the total number counted with 
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multiplicities: the length of the scheme in over which the union of the fibres sit.) 
Write Q = (Bf^ii*Qi, where Qi is a sheaf supported on Xf. . 
Since £ is torsion free we have the sequence 

inducing 

^ Hom(r(-n + 1),^) ^ Hom(r(-n),^) ^ Rom{T{-n),£ \xtj- 

Therefore if in (pip is zero on . it comes from an element of the first group in 
the above sequence. Thus, by reducing n if necessary, we may assume that 4>\xt- 0) 
so that the rank of Qi is at most r — 1 on its support. 

Since £ is reflexive on a smooth variety, it has homological dimension at most 
one, i.e. £xf{£,J-) =0 Vi > 2, for all coherent sheaves T. Also T[—n) is locally 
free, so from ( |4.11j ) we see that £xf{Q,T) =0 Vi > 2, and Q has homological 
dimension at most one. Therefore it is supported in exactly codimension one; i.e. 
the rank of each Qi is at least 1 (since there are no reducible fibres). That is 
1 < Tj < r — 1, where rj = rkQj. 

Now the slope stability of T\xt-i and the sequence 

T\xt^ £ \xt^ ^ Qi^O, 
imply that the slope of Qi is less than that of T\xf -- That is, 

ciiQi).u;\xt, < -ci(T) .LUi.LO, (4.12) 



where uji is the pullback via vr of the standard Kahler form on . 

By the Grothendieck-Riemann-Roch theorem (or more elementary considera- 
tions), ch{i^Qi) = L^ch{Qi), where i : Xt- ^ X is the inclusion of a fibre (in 
particular it has a trivial normal bundle), and ch is the Chern character. Therefore 

ciii^Qi) = Viivi and C2{i*Qi) = -i*ci{Qi), 

since ci(<.*Qi)^ = 0. (The pushforward on cohomology t=K increases the (complex) 
degree by one in cohomology; it is Poincare dual to the inclusion on homology.) 
Combining this with ( 4.11J ), and denoting the total Chern class by c, we have 



ci£) = ciT{-n))f[cii,Qi) = 

i=l d d N 

( l + ci(r) -rna;i + C2(r) - {r - l)nuJiCi{T) ] • I 1 + "^^^1 - ^ t*ci((5j) 

^ ^ \ j=i i=i / 
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up to second degree in cohomology. Since £ and T have the same Chern classes, 
this gives 



C2 



{8 ) - C2{£) = ^{ri) -rn\uji+ I ^(r^) - (r - l)n wiCi(r) - ^ L*ci{Qi). 



\j=l / \i=l / 1=1 

The degree one piece thus gives 

d 

Tj = rn. 



1=1 



which with ( [4.12 ) yields 

d 

'^ci{Qi) .Lo\xt^ < nci{T) .LOi .LJ, (4.13) 



i=l 



unless the number of fibres d is zero, in which case both sides vanish. Taking the 
cup product of the second order piece with uj, however, gives 

d 

nCl(r) .Wl .W - ^Ci(Qi) .Wlxt^ = {C2{£) -C2{£)).L0, 
i=l 

which, by ( [4.10|), is nonpositive. Therefore ( [4.13| ) cannot hold, and so d = = n, 
(5 = 0, and ([4.1l| ) becomes 

^£ ^0. 

Thus the Chern classes of £ and £ are the same (they are both equal to that of T) 
and £: ^ ^ ^ T. q.e.d. 

Examples 

• Consider the Calabi-Yau 3-fold that is a smooth (2,2,3) divisor in a product of 
projective spaces, 

^2,2,3 C X X 

The projection vri to the first P^ exhibits X as a K3 fibration, with fibre a (2,3) 
divisor in Pi X P^ . In turn this fibre is a double cover of P^ branched over a sextic 
- the zero locus of the discriminant of the quadratic on P^ (which has coefficients 
that are cubics on P^) defining the K3 fibre. It is well known that the pullback of 
the tangent bundle of P^ is an isolated slope stable bundle on any such K3 with 



respect to the pullback of the polarisation on P (the proof in [DK 9.1.8 works even 
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for the singular covers). So taking the polarisation 7rlO{N) 7r30(l), N ^ 0, in 
the obvious notation, we see that 

is slope stable and unique in its moduli space of sheaves, giving a holomorphic Cas- 
son invariant of one. The same is true of the pullback of TP^ to a more general 
double cover of X p2 branched over a smooth (4,6)-divisor. 

• On K3 X T^, fixing determinants to be trivial in the direction, the method 
of the proof of the theorem shows that, given an isolated slope stable bundle on 
K3 with gcd(r, ci .u>, ^cf — C2) = 1, we get a holomorphic Casson invariant of 
by pulling the bundle back from K3 and twisting by line bundles on whose rth 
power is trivial. (Notice that here we are taking a polarisation with the fibres 
large, the opposite of FMW| .) It would be especially interesting to study the higher 



dimensional moduli spaces. The parts of the moduli space arising as pullbacks from 
K3 are smooth and the corresponding invariant would be the Euler characteristic 
of the moduli space, to within a sign. Generating functions of these give modular 
forms, by the work of Vafa and Witten |VW], so it would be interesting to compute 



the corrections given by other parts of the moduli space. For instance, the next 
section will show there are no corrections for the 2-dimensional moduli spaces and 
we get a Casson invariant 

X{K3xT^) = r\(MK3). 

• It would be fashionable, and by now almost expected, to find modular forms 
arising from the Casson invariants on more general Calabi-Yau manifolds. An easy, 
cheating, way to produce them is to take the moduli spaces of stable bundles on Fano 
surfaces in | VW| , and push these forward to give torsion sheaves on any Calabi-Yau 



3-fold that is modeled locally on the total space of the canonical bundle over the 
Fano. Deformation theory ( p.42| ) shows that all sheaves in the same component 
of the moduli space are of the same form, so the holomorphic Casson invariant is 
the Euler characteristic of the moduli space if it is smooth, and some appropriate 
modification of it if not. (In fact this gives a way of rigorously defining the Euler 
characteristics of moduli spaces in [ VW] so long as there are no semistable sheaves.) 



Thus generating functions of the invariants (for Chern classes on the Calabi-Yau 
that are pushforwards from the Fano surface) do indeed give modular forms in the 
cases studied in [lVW| . 



We can also consider the two examples of Mark Gross [Gr], piu|. Let Ei 



be the trivial rank four bundle 0®*^ over P\ E2 = 0(-l) © 0(1), and 
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Pi = ¥(Ei). Let the Xi be anticanonical divisors in Pi (Gross shows they can be 
chosen smooth). They are clearly both fibred vr : Xj ^ by K3s that are quartics 
in P^. Their second cohomologies are generated by t := ci(Op(£;-)(l)) and the fibre 
class / := 7r*uJi. We use a polarisation t + Nf, N ^ 0. 

The Xi are in fact diffeomorphic via a diffeomorphism taking t and / on Xi 
to t and / on X2. They are not, however, deformation equivalent as projective or 
even symplectic manifolds (by work of P. M. H. Wilson in the first case, and Gross 
and Ruan in the second piu| ). We might hope to be able to prove the projective 
statement (or even both if Tian's work mentioned in Section 3 can be developed 
to give symplectic invariants) using the holomorphic Casson invariant. The most 
obvious bundle to take is, in the first case, 

Tr*TF'^ Xi. 

Proposition 4.14 The restriction o/TP^ to the generic Xi — > P^ is, on restriction 
to each fibre, slope stable (with respect to the Fubini-Study Kdhler form), isolated, 
and unique in its moduli space. The holomorphic Casson invariant of any Xi, for 
sheaves of total Chern class 1 + 4t + Qt'^ + At^, is 1. 

Proof While it is well known that TP^ is stable on the generic quartic this is not 
enough for us - it is a priori possible that the unique stable bundle on each quartic 
is generically TP^ but something else on a closed subset of quartics. We need to 
show that it is stable on each quartic S" in a generic one-parameter family Xi. The 
other statements follow by the usual methods if we can show that TP^ is stable, 
since it is locally free and so satisfies Serre duality even on the singular quartics. 

So fix a (possibly singular) quartic 5 C P^. Suppose firstly that TP'^js' is (slope) 
destabilised by a rank two subsheaf. Then the quotient map gives us a sequence 

rp3(-i)|5 ^ /: ^ 

on S, with C a torsion free (without loss of generality) rank one sheaf. C is generated 
by its sections since rP^(— 1) is, and of degree less than or equal to 1. If £ were 
trivial then the dual of the above sequence would give r2p3(l)|5 a section which the 
restriction sequence from P^ shows it does not have. Therefore C must have degree 
1 and at least 2 sections to be generated by them. 

Sections of C vanish on a degree one curve in S* C P'^, which must therefore be a 
line. Taking two sections of C we get two homologous distinct P^ s in 5" which is a 
contradiction if 5 is smooth since they must have self-intersection —2 by adjunction. 

As Adrian Langer pointed out to me this makes no sense if S is singular; never- 
theless the argument goes through as there are no 1-parameter families of P^ s in 5 
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for S a fibre of a generic Xi (for instance we can therefore assume that S has only 
ODP singularities). 

Since we can assume S irreducible and reduced it is now sufficient to consider 
rP^(— f) being destabilised by a rank one subsheaf. Dualising, we get a sequence 

np3{2)\s ^ c ^ 0, 

for some other torsion free (without loss of generality) rank one sheaf C. Again this 
shows that C is generated by its sections, and is of degree < 2 this time. But degree 
2 curves in are curves of arithmetic genus 0, and a similar argument applies. 

Therefore, by Theorem ([4.§|), 7r*TP'^ — > Xi is the unique stable sheaf in its 
moduli space with total Chern class 1 + 4t + + At^ . Thus we have a holomorphic 
Casson invariant of one, for the generic Xi, and therefore for all Xi by deformation 
invar iance of the holomorphic Casson invariant, q.e.d. 

Unfortunately this does not distinguish the XiS since for the corresponding (un- 
der the diffeomorphism) Chern class 1 + 4t + 6t^ + on X2 the invariant is also 
one, given by the relative tangent bundle down the fibres of P(-E'2): 

T^F{E2)\x, ^ X2. 

So to distinguish the Xi s we have to consider two dimensional moduli spaces on the 
K3 fibres. 

The fibre dimension two case 

To study bundles with d = 2 dimensional moduli spaces on the fibres we follow 
the same method as before, considering sections of the fibration of fibrewise moduli 
spaces - the "Mukai-dual" 3-fold — > P^ constructed by applying Mukai duality 
(replacing a K3 by a 2-dimensional moduli space of sheaves on it - another K3) 
fibrewise as in Theorem |4.5| . We will show that A4 is Calabi-Yau. 

In this two dimensional situation a new feature can occur, namely the sections 
can change homology class. However, just as introducing codimension two singular- 
ities and unstable fibres drives C2 higher, we shall find that changing the section to 
one of higher degree does the same (where degree is measured appropriately) . Thus 
for sections of the smallest possible degree this cannot happen in a fixed moduli 
space. 

Relating the deformation theories of the bundles and the sections will show the 
Casson invariant of X is equal to the appropriate Gromov-Witten invariant of A4. 
Thus we see Gromov-Witten invariants arising, but not in the way we might have 
expected (with rank 2 bundles corresponding to curves via zero sets of their sections), 
and in fact for arbitrary rank. 
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Throughout this section we will work with K3 fibrations whose singular fibres 
have only single ODPs. Many of the results are true for arbitrary reduced irreducible 
fibres (and in fact all of them should be, but I cannot prove it). We start with some 
technicalities improving on Theorem |4.5|. 



Theorem 4.15 Fix {X — > P^,r, q) as in Definition with d = 2 and r > 2 



Then the moduli space — > is smooth with fibres Mt having only ODPs as 
singularities. The singular points represent the reflexive non locally free sheaves on 
Xt; all other points correspond to vector bundles. 

Proof These results were conjectured in an earlier version of this paper based on 
comparing monodromy in — > around singular fibres with the corresponding 
monodromy in X — > P^ (using Mukai's natural isomorphisms between the cohomolo- 



gies of the fibres Xt and A4t as in ([HL] 6.1.14); see Theorem 4.23 below). Since then, 
however, a remarkable proof of the smoothness of A4 using Fourier-Mukai transforms 
has appeared | BrM| ] (this applies directly in our case, even without assuming RDP 



fibres: the assumptions on Chern classes (|4.3D make the fibrewise moduli spaces of 



"Fourier-Mukai type" in the terminology of [BrM] - the Mukai vector of the sheaves 
is primitive of square zero). The remaining results can be deduced from standard 
theory of reflexive sheaves on ODPs, and I would like to thank Akira Ishii for ex- 
plaining this theory to me. 



From Theorem 4.5 all sheaves in A4 are reflexive, and locally free if supported 
on a smooth fibre. So consider a reflexive, non locally free sheaf £" on a singular 
fibre Xf. As in the proof of Theorem |4.5| , consider £ = tt*£/ torsion on the minimal 
resolution vr : — > Xf. We proved that £ was simple and had d{£) < d{£) = 2, so 
that it sits in a moduli space of dimension zero: 

Ext^(f',^) = 0. 

In fact from equation ( [l.T] ) we see that the local first Chern class satisfies c\{£,y) = 
—2, where y is the ODP (for instance, dimension considerations force it to be between 
— 1 and —3, and it must be even since this is the only ODP, and the intersection 
form on K2> is even). 

Thus standard theory of reflexive modules (see e.g. Q Example 3.2) implies 
that £, restricted to the exceptional — 2-sphere Z C Xt, is 

o;7^eOpi(i)eOpi(i). 

The local deformations of the corresponding reflexive module over the ODP are then 
given by (Q Theorem 4.3 (2)) pushing down bundles whose restriction to Z is any 
deformation of 

0;7^©0pi(l)e0pi(-l) (4.16) 
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(or Op7^ © C)pi(— 1) © Opi(— 1)) but this gives the same local reflexive module). 
Pushdowns of deformations of the above bundle give the 2 dimensional local moduli 
space of deformations of the sheaf. This has an ODP (the original reflexive sheaf) 
away from which the sheaves are all locally free (Q Theorem 4.6), corresponding 
to the deformation 0®^ of ( [4. 161 ) on the resolution. 

Now H'^{£nd£) =Rom.{£nd£, Kxt)* by Serre duality, and this is C {Kx^ is 
trivial and £nd{£)* = £nd£ since it is reflexive - it is the pushforward of the 
restriction of itself to the smooth locus of a normal surface) . (Thanks to Akira Ishii 
for this argument.) 

The local-to-global Ext spectral sequence yields 

^ H\£nd£) Ext^(^,^) ^ H°{£xt\£,£)) H\£nd£), 

where the last term is all trace H'^{Oxt ) — ^ that survives in the spectral sequence 
for Ext^(£',£'). Thus the penultimate map is onto, and global deformations of £ 
map onto local deformations £xt^[£,£) of the module over the ODP. But both have 
dimension 3 (global deformations of £ are the deformations of a singular point in 
a surface Mt in a smooth 3-fold A4, so have Zariski tangent space of dimension 
3), so we get a local isomorphism from the global deformations to the local model 
described above. In particular we see that there are locally free deformations of £, 
the moduli space is 2 dimensional, and the non locally free sheaf is isolated in its 
moduli space and represents an ODP. q.e.d. 

We need to fix an appropriate polarisation on ^A ^ F^. In fact, all that will 
concern us will be its restriction to the fibres, so we consider a polarised K3 surface 
S with at worst an ODP singularity y G S, and Ms a moduli space of semistable 
sheaves of fixed determinant such that rank and degree are coprime. Let H he a 
generic smooth hyperplane section of S missing its singularity y, p : 5 x A^g — > Ais 
and q : S X Ms S he the projections, and let £" be a (local) universal sheaf. 
Then there is an obvious class [£ndQ{£\f{)] in the if -group K^{S x A4s) which is 
just £ndQ£ if £ is locally free, and more generally the kernel of the trace map on the 
restriction to H of the tensor product of a finite locally free resolution of £\s\^s^y-^ with 
the dual complex. This is uniquely defined even though £ is only defined locally up 
to tensoring with a line bundle pulled back from A4. 

Likewise the push down 



p^ : K^{S X Ms) K^{M 



s) 



([tIL| 2.1.11) isp!=p* — R^p^ + B?p^, or p^ on a finite resolution by sheaves with 



no higher cohomology (this always exists |Hal] III 2.7). Then the result we need is 
the following. 
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Lemma 4.17 In the above set-up and notation, the determinant line bundle 



(detp! SndoSlH)* 

is ample on Ms- 

Proof Fix a point x G and let |{x}xA4s denote q*Ox in the K-group (resolve 

by locally frees away from y and restrict to x). Let x = x{£\h) = x(^) ~ x(^(~l))- 
Then it is a result of Jun Li (0, Section 8.2) that 

detp! U£\{x}xMs - 

is ample on the moduli space of slope stable sheaves (which for us is all of Ms) 
for S smooth. But the proof goes through for surfaces with isolated singularities 
for rank and degree coprime: for a fixed bundle £ the restriction to the generic 
curve in |0(a)|, a ^ is semistable (Flenner's theorem ( ||HL|1 7.1) applies to normal 
varieties) and so stable if a is coprime to the rank, and the generic curve misses 
the singularities and is smooth. Now proceed as in ( pIL(| 8.2) to deduce ampleness 
of Jun Li's line bundle on the locus of locally free sheaves. As the non locally free 



sheaves are isolated in the moduli space ( [4.15| ) this is sufficient 



Thus it is sufficient to show that the two line bundles have the same first Chern 
class. 

But by Grothendieck-Riemann-Roch we have 
ClPl{x£\{x}xMs - r£\H) = P*[ch{x£\{a:}xMs " ^"^1^) Td(i?)] ^ 

= p*{-rch2iE\H)) + ci{£\{.x}^Ms) (x-^r{2-2g) 

where g is the genus of H C S. Therefore x — (2 — 2g) is the degree of f i.e. 
ci{£), so we obtain 

ciP\ix£\{x}xMs - ^^\h) =p*{-rch2{£\H) + ]^ci{£\Hf ) 

1 1 

= p*{rc2{£\H) - -{r - 1) ci{£\Hf) = -^P* ch2{£ndo£\H). 

But since chi{£ndo£\H) = 0, Grothendieck-Riemann-Roch equates the last term 
with ^ci{{p\{£ndo£\H))*)- q.e.d. 

So consider again the relative moduli space — > P^, with a (local) universal 
sheaf T. Let p be the projection p : XxpiAi M, and pick a smooth hyperplane H 
of X, flat over P-^. Then the first Chern class Q of the line bundle (detpi {£ndQT\H))* 
is, by the above lemma, ample on the fibres Mt- 
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So denoting by loi the pull-back to M of the Fubini-Study form on F^, NiOi + ^ is 
ample on Ai for N sufficiently large. N will be unimportant for us (we are concerned 
with sections of — > P"^ which all have fixed degree measured against wi), so we 
fix one such N and measure the degree of sections against the above polarisation. 

In the following, for any section t : — > we will denote by i the induced 
section i : X ^ XxmM. 



Theorem 4.18 Let {X,r,Ci) satisfy Definition 4-4- Denote by T a universal sheaf 
on Xxjf>iA4 as produced by ( [^.^ j, and choose a section a of A4 ^ ¥^ of strictly 
minimal degree with respect to the polarisation Nui + 0, described above. Then all 
stable sheaves with the same Chern classes as a* T are pull-backs of T by sections 
in the same homology class, and these are locally free. 

Remarks The theorem should be true in greater generality, without the assump- 



tions of Definition 4.4. For fibrations with worse singular fibres one can extend the 



proof, getting slightly weaker results. In fact one can show, using |BrM] to study the 
obvious Fourier- Mukai transform on a singular fibre Xf x Mt, that smooth points 
of the moduli space are in one-one correspondence with locally free sheaves. Un- 
derstanding when the singularities of A4t are isolated is more difficult, and stops 
one proving the ampleness (f4.17 ) of Jun Li's line bundle. But in examples this is 



obvious, so the results still apply. 

As mentioned earlier the theorem also applies to 3-folds fibred by surfaces with 
negative canonical bundle, and arbitrary base curves if we take account of the fun- 
damental group. The iCS-fibred case is most relevant to us, however. 

Proof We want to study an arbitrary stable sheaf £ with the same Chern classes as 
a* T. We follow the proof of Theorem |4.8| , adapting the argument where necessary. 

As in ( [4. 81) the restriction of the double dual £ to the generic fibre must lie in the 
same moduli space as T, giving us a rational (and so regular) section s : ¥^ A4. 
Again, as before, this gives us a sequence 

^ rr(-n) ^£ ^0, 

for some n, with Q = (Bf^iL^^Qi supported on a finite number of fibres {Xt-}f^i, 
rank (Qi) = ri, 1 < rj < r, and (f)\xt. 0- 

Computing Chern classes as in (4.8) gives, for ci, 




ci(a* T) - ci(r T) = yin) - rn cui, (4.19) 
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which, with the stabihty inequahty (4.12), yields 
d 



i=l 



1 



n 



L^i + -(ci(a* T)-ci{s*T)) 



.ci(s* T).uj, 



(4.20) 



with the inequality strict if d 7^ 0. For C2 we obtain 



d / d \ 

C2(?) = -5^^*ci(Qi)+ J^(r,)-(r-l)nU;i.ci(rr) + 02(3* T). 

i=l \t=l / 



Proceeding as before, with ( ^.201 ) this yields 

> {c2i£)-C2{£)).UJ > 



1-7) [ci{^*T)-ci{s*T)].ci{s*T).Lo + [c2{s*T)-C2{rT)]uj, 

with a strict inequahty for d ^ 0. 

Now [ci {a* T) - ci (s* T)] ^ = from (|1|) (this also shows that the inequality 
is independent of twisting T by a line bundle on P^, as it should be), so we may 
rewrite the first term in the above inequality to arrive at 



> (r - a*) 



1 



C2(r)-- 1-- ci(r) 



The idea now is that the quantity in the square brackets is the discriminant (| 
3.4) of T (divided by 2r), which, by stability, should be positive in some sense. In 
fact, letting H he a. smooth hyperplane section dual to w, flat over as before, we 
have 



> / (cT* — s*) uj . c/i2 {Sndo 



X 



T)= f {d* - r) ch{£ndoT\H) Td(Tp 
Jh 



Here p denotes the projection p : H XpiAd ^ Ai, with relative tangent bundle the 
class Tp in K-theory, and the other terms in (ch . Td) do not contribute as chi = 
and cho Td(Tp) is the same under s* and a*. Also \h is meant in the sense of 
multiplication by O^Xpi in K-theory, as before. 
Thus, by Grothendieck-Riemann-Roch, we have 

> / {a* - s*)p^{chi£ndoT\H) Td{Tp)) = [ {a* - s*) ch {p, SndoTln), 
where p\ = — R^p^. Thus, finally, 

0> / {s* -a*) ci{{detpi£ndoT\H)*). 
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Letting = ci[{detp\ SndoTlff)*^ , which is ample on the fibres, A^i^i + is a 
polarisation for N ^ 0. s and a are both sections of — > and so have the same 
degree with respect to loi, and so we arrive at 

0> / {s*-a*)n= [ {s*-a*) (Ntoi + n). 

Thus if, as in the conditions of the theorem, a is of strictly minimal degree then 
we must have s and a in the same homology class, and, as the inequality is now 
non-strict, d = = Q. By (|4T9| ) n = 0, and so s* T ^ ^ ^ £. 



Finally, sections s : ^ A4 miss the singular points of fibres where the deriva- 
tive of the projection to vanishes. Thus s takes its image in the smooth points 
of which correspond to locally free sheaves on the fibres by ( [4.15| ). Thus s* T is 
locally free, q.e.d. 



Corollary 4.21 Under the conditions of Theorem ^.18 , the holomorphic Casson 
invariant of X equals the algebraic Gromov-Witten invariant GW^([(t]) of A4 (as 
defined in |LT|| , [p^ ] j counting curves in the same homology class as a. 



Proof Let J\f denote the moduli space of curves in the homology class (which is 
isomorphic to the space of stable maps as the curves are sections - they can have no 
bubbles in fibres by the assumptions on degree - and so regularly embedded). Use 
s : AAxP^^TVxA^to denote the universal curve, with s : M x X ^ M x MxjpiX 
the induced map . Let the pull-back of the universal sheaf on Xx piA4 to TV x Xx piAi 
be denoted by £. Then the sheaf s*£ on TV x X exhibits TV as the relevant moduli 
space of sheaves on X as produced by the above Theorem, depicted by the diagram: 

s*£ £ 
i _ i 

J\f X X A Af X MxpiX 

iP i TT 

A/" X pi A MxM 

iq i P 

TV TVxpi. 

We must show that the tangent-obstruction functor of the sheaves is the same as that 



of the curves in the Gromov-Witten theory of |LT]. Since the curves are sections they 
miss the singularities of the projection A4 and so, by the above discussion, lie 

in the open set of A4 corresponding to locally free sheaves on fibres, which therefore 
satisfy the Serre duality (|4.lD on their Exto s. Thus the only non-zero such Exto is 
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Ext^^(^f, ^t)o (for Tt a sheaf on the fibre X^), which is the fibrewise tangent space 
to ^ P"*^ at Tti i.e. the normal bundle to any section through Tt- 



For smoothly embedded rational curves the deformation theory of |LT| reduces 
to K^Qi^Us ^ I, i = 0, 1, where is the normal bundle to s and X is an arbitrary 
0^-module. This is then 

R'q4s*Tp) = R'q4s*£xtl{£,£ (E)TT*I)o) = R'q4£xtl{s*£,s*£ (^p*I)), 

where for the last equality we have base-changed around the square in the above 
diagram using the flatness of £ and (£'xf^)o (since the other (£'xf^)os vanish). By 
the Leray spectral sequence this yields £xf^p{s* £ , s*£ (8) (g op)*T)o, i = 0,1, which 
gives the deformation theory of the sheaf moduli problem, as required, q.e.d. 

Remark Degenerating the base to a curve with one node degenerates A4 (which 
we show below is Calabi-Yau) to a normal crossings space of two "Fanos" (their an- 
ticanonical bundles are effective) joined across a common anticanonical divisor S - 
the fibre of M over the node. Then the Tyurin-Casson invariant picture sketched in 
(p.7| ) becomes, in terms of the sections of — > P^, a formal picture mentioned in 
|T1| (where it was also motivated by relating bundles to curves, but in that case via 
zero sets of rank 2 bundles). Namely, to count curves in such a singular Calabi-Yau, 
one should count those in each Fano component which meet in the "boundary" S. 



Analogously to the Tyurin picture we find |T1| that the image of the map taking 
a curve in one Fano component to its intersection with 5, which generically lies in 
the complex symplectic space Hilb"(S'), is a complex Lagrangian. (Here n is the 
intersection number of the curve with S.) Intersecting the two Lagrangians should 
give something like the GW invariant of a smoothing of the Calabi-Yau. It would 
be nice to rigorise this and the Tyurin-Casson invariant (of course one is a special 
case of the other, by considering the case of ideal sheaves of curves). 

We next show that A4 is in fact a Calabi-Yau 3-fold. I have since discovered 
this was conjectured to be true many years ago by Mark Gross. The result has now 
also been given a new proof in iBrM], using Fourier-Mukai transforms, without the 



need for assumptions on the singular fibres of X. Notice that although ( 4.18| ) did 



not hold for rank r < 1, this result does, provided that A4 is smooth and sheaves 
supported on singular fibres without locally free resolutions are isolated. Thus, in 
the case that the fibres Xt are themselves elliptically fibred, we may take as A4t 
the moduli of torsion sheaves that are rank one, degree zero sheaves supported on 
elliptic fibres (so that Ait is a compactified Jacobian of — > P^). In a hyperkahler 
rotated complex structure (and for some appropriate choice of "B-field" ) this should 
be the mirror K3 to Xt ( [pYZ| 4.1) and is a obtained from X by fibrewise mirror 
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symmetry and hyperkahler rotation. However is not the mirror of X (its Hodge 
numbers are not flipped for instance; see below), though it is "T-dual" to X. 

Theorem 4.22 Let A4 be a relative moduli space of slope stable sheaves on X with 
data (^), d = 2, r > 2, and single ODP singularities in the fibres of X P"*^. 
Then M is a Calabi-Yau 3-fold. 

Proof As before consider M to be the moduli space of torsion sheaves {it)if£, where 
£ \s a, sheaf on Xt <Z X. Let T denote the universal sheaf on X XpiTW, and let 
i : X^^^l^A ^ X X Mhe, the inclusion and p : X x M ^ M the projection. Then, 
as M. is smooth ( 4.15| ), it has a tangent sheaf given by £xtp{i^T, l^T). 

The holomorphic 3-form is the pairing on the tangent bundle given by the cup 
product pairings 

£xtp{L^T, i^T) (g) £xtp{L^T, i^T) £xtp{L^,T, i^T) £xtp{i^T, i^T) = 0;^, 



in the spirit of Mukai's symplectic structure for moduli spaces on K2> surfaces [Mul]. 
Notice its dependence on the holomorphic 3-form on X is through the final isomor- 
phism. To show that this pairing is non-degenerate everywhere and that M is 
Calabi-Yau it is enough to show it does not vanish on any divisor in M.. But by 
( 4.15 ) it is enough to show this at points of M corresponding to locally free sheaves 
since their complement is isolated; these then satisfy Serre duality on their Exts. As 
then all of the sheaves in the above pairing are locally free and we can localise at a 
point to get, at the level of tangent spaces, the cup product pairing 

Ext^(J^, F) (g) Ext\T, T) Ext^(.F, T) Ext^(.F, T) = C 

at the point = {Lt)*£ £ -A4, where £^ is a sheaf on the fibre Xt. 

So now to show the pairing is non-degenerate it is enough to show, by Serre 
duality, that the pairing 

Ext^(J^, J^) (g) Ext^(J^, J^) Ext2(J^, J^) 

is onto. By Lemma |3.42| above the inclusion Xt C X induces a long exact sequence 

. . . ^ Ext^^ {£, £) Ext^(.F, ^ P^ «) Ext^/(f, <S) ^ . . . , 

which yields, for Xt a K3 and £ stable and satisfying Serre duality, 

^ Ext\^{£,£) Ext3f(j^,J^) ^ TiPi ^ 0, 

and 

^ Ext^^(f,f) ^ Ext^(J^,J^) ^ TtF^ (E)Eyit]^^{£,£) 0. 
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The pairing respects these sequences in the obvious way. Firstly the cup product 
of the image of two elements of Ext (£",£") C Ext^(jr, JF) is just (the image of) 
the cup product on Xt, with image in Ext^^ (£:,£:) C Ext|-(J^,j^), and this IS non- 
degenerate. Secondly pairing an element x of Ext^(^, ^) with (the image of) an 
element y of Ext^^^ {£, £) and projecting Ext|.(:r, T) ^ ® Ext^^^ {£, £) gives the 
same as projecting x to T^P^ and tensoring with y. Again this is onto, so we are 
done, q.e.d. 



We now study the cohomology of such an A1, putting Mukai's isomorphism of 
Hodge structures (|HL] 6.1.14) between Xt and M.t into a family XxpiA^. We avoid 



problems with the singularities of fibres of X by working on X x M. 

Fix X and a rank and Chern classes satisfying the conditions of Theorem 4.18, 
pick a universal sheaf £ on X x M., and let vr and p be the projections to X and M. 
respectively. 

Then define maps 

/ : H*{X;C) H*{M;C), f : H*{M;C) H*{X;C), 

by 

f{c) = (ch" {£) y^Td{X xM).7r*{c)), f'{c) = 7T,(-ch{£)^/Td{X x M) ./(c)), 

where c/i^ = ^(— l)*c/ij. 

Theorem 4.23 f o f is the identity. 

Proof We simply mimic the exposition of Mukai's proof for K2> surfaces in ( pIL(| 
6.1.13). Label two copies oi M. by Ali and M2-, and denote by the pull back of 
the universal sheaf on X x ^Ai to X x Ali x ^A2■ We must transfer a class c from 
Ml to X via /', then back to M2 via /. Pulling everything back to X x Mi x M2 
via the commutativity of various push-down pull-back squares, and pushing down 



X before M\ (as in |HL] 6.1.13), we can reduce to the following diagram 



X x Ml x M2 
iP 
Ml X M2 
Q2 y\ qi 
M2 Ml , 



arriving at 



ff'{c)=q2,p, -ch''{£i)ch{£2WTdiMi X M2) TdiX).ql{c) 
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where we have supressed some obvious pull-back maps for clarity. By multiplicativity 
of the Chern character this yields 



Q2h 



p^{-ch{Kmm{£i,S2) Td{X))y^Td{Mi x M2).qKc) ■ (4.24) 



Now p*(c/i(R7Yo?7i(<Si, 1S2) Td(X)) = ch{'RTConip{£i,£2)), by the Grothendieck- 
Riemann-Roch theorem. But we know that 

Ext*y(.Fi, =0, Vi, (4.25) 

for sheaves J^i J-2 in A4 unless both are non locally free on their support (where 
again we are considering to be a moduli space of torsion sheaves supported on 
fibres Xt). This is because if either is locally free then the Serre duality (4.1) holds 
on the fibres and the usual arguments go through ( |HL| ] 6.1.8, 6.1.10) to give the 
vanishing of all Ext^^s on the fibre; thus by ( p. 42 ) all Ext^s vanish too. 

So the Ext*s {i < 3) vanish outside a subvariety of codimension three (the union 
of the diagonal XxAcXxAlixA^2 and the codimension six ( [4.15 ) locus of pairs 
of non locally free sheaves) for all i < 3, from which it follows ( |Mu2 | 2.26) that 

5x4(^1,^2)^0, i<3, 

and £xtp{£i, £2) is concentrated on A. On A we have Ext^{T,T) = H'^{Ox) via 
the trace map, inducing an isomorphism 

Tinomp{£i,£2) [3] ^ £xtl{£,£) R%OxxA = Oa- 
Again by Grothendieck-Riemann-Roch we have 



c/i(t*OA)\/Td(yWi X M2) = L*ich{(DA)), 



so plugging everything into ( |4.24| ) (and noting that the shift by [3] introduces a 
minus sign in ch) gives 

ff'{c)=q2 4ch{0A)qt{c))=c. q.e.d. 



Corollary 4.26 Let M. he a relative moduli space of slope stable sheaves on X with 
data ^-4) , o-nd d = 2, r > 2. Then the groups = ®j„j=fc H^'^ are isomorphic on 
X and A4; thus their Hodge numbers h^'^ are the same. 
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Proof Since the Fourier- Mukai machinery of |BrM] now gives a quick proof of this 
result we will only give a sketch of our previous argument. 

Note that all we want to do is reverse the roles of X and A4 , using the universal 
sheaf on XXf,iAi, restricted to fibres {xt} x Mt, to exhibit X as a fibrewise moduli 
space of sheaves on A4 and deduce ( [4.23| ) that f o f = id. 

Pick a smooth fibre Xt and consider the universal bundle on Xt x Mf Firstly, 
by results of Mukai, (see e.g. [Br2| Theorem 1.1), the corresponding Fourier-Mukai 



transform is an equivalence. By using the corresponding transform for Hermitian- 



Yang- Mills connections in differential geometry (see for example | BBH |), one can 
show that the transform maps slope stable sheaves to slope stable sheaves, making 
Xt a moduli space of slope stable sheaves on A4t of "Fourier-Mukai type" (that is the 
Mukai vector is primitive and of square zero; equivalently, gcd (r, ci . oj, ^cf — C2) = 1 
for an appropriate choice of polarisation). 

Thus even on singular fibres semistable sheaves are stable and so simple, so long 
as we can show that X really does parametrise sheaves on A4 over these fibres; i.e. 
we need to show that the universal sheaf on X x is flat over X. 

But this follows by mimicking the argument of ([ [Brl | Lemma 5.1) one dimension 
up on K3 fibrations instead of elliptic fibrations. (All that |Brl| uses is the fact that 
the sheaves parametrised by A4 are flat over A4 and have a locally free resolution of 
length 2 on X. In our case flatness over A4 is again immediate, and since the sheaves 
are reflexive on the fibres by Theorem |4.5| they have depth 2 and so homological 



dimension 1 by the Auslander-Buchsbaum theorem ([HL| 1.1).) 

Using (Gieseker) stability we get the condition ( |4.25 ) so that the proof of Theo- 
rem 4.23| goes through with X and A4 exchanged. 

Since ch and Td are of pure {p,p) Hodge types, the I^s are preserved. Calcu- 
lating their dimensions in terms of h^'^, h^'^ and /i^'^, by symmetries of the Hodge 
diamond, we see that these three numbers are preserved, q.e.d. 



Theorem 4.23| is of course the fibrewise Fourier-Mukai transform of [ BrM | at the 



level of K-theory. Theorem 4.15 can also be interpreted in terms of this transform; 
the transform of ©[o-]) where [a] is the image of a minimal degree section oi Ai ^ 
F^, is just a*T. Thus one might try to use [BrM| to study moduli of sheaves on 
X; the difficulties are then displaced to understanding when the transform gives 
stable sheaves, and not more exotic elements of the derived category. Nonetheless, 
one might try to study moduli by transforming the structure sheaves of connected 
curves in a class that intersects the fibre class once. The singular nodal sections 
with components lying in fibres will give rise to fibres on which instabilities and 
singularities of the sheaves lie. 
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Examples 

We have already mentioned how the two dimensional results apply to K3 x T^. 
Now we again study Gross' examples, this time considering two dimensional moduli 
spaces, doing a family version of an example in ( |HL| ] 5.3.7) and extending the proof 
of stability to all quartics in P^. 

Proposition 4.27 Fix any irreducible, reduced quartic surface 5 C which con- 
tains no family of curves of arithmetic genus 0, and a closed point x ^ S. Then the 
rank two sheaf E = defined by the sequence 

O^E^ H\Ml)) ^ ^.(1) ^ 0, 

is slope stable with respect to the restriction of the Fubini-Study metric. Ex is a 
bundle for x a smooth point of S, and the above gives an isomorphism between the 
appropriate moduli space and S. 



Proof The dual E* of such a sheaf is generated by its sections, and a destabilising 
rank one (torsion free, without loss of generality) quotient sheaf C is therefore also 
generated by its sections and of degree < 2. Therefore the proof of ( 4.14 ) applies to 
obtain a contradiction, q.e.d. 



So in this case, for the two Gross 3- folds Xi, the relative moduli spaces Mi are in 
fact isomorphic to Xi for the generic Xi with fibres satisfying the above conditions. 
(We need only calculate the holomorphic Casson invariant of the generic Xi, by 
deformation invariance.) It is easy to see that the polarisation on Mi is that on X 
plus Nloi, since this gives the right fibre polarisation on ^At for the generic fibre Xt 
of Picard number one. Unfortunately this example only satisfies the conditions of 
Definition not Definition 4^, so Theorem 4.18| does not really apply. But the 



extra conditions of Definition 4.4 were only used to show that Jun Li's line bundle 
( 4.17 ) was ample on the singular fibres oi M ^ P^, and that the smooth points of 
the fibres corresponded to locally free sheaves, both of which, as mentioned earlier, 
should be true in general anyway. I cannot prove it in the general case, but it clearly 



holds in this example, so we may use Theorem 4.18 



Thus the holomorphic Casson invariants are given by counting the number of 
lowest degree sections of 



[Xi 



C (P(S,) ^ P^) 



with respect to the form t = 
subbundle 0(1) C E2 = 0(- 



ci(Op(£'.)(l)), which is ample on the fibres. For X2 the 
-1) ffi ffi e 0(1) gives a section of ¥{£2) P^ which 
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lies in X2, is of lowest degree (—1) with respect to t, and is unique. Thus we get a 
Casson invariant of one, counting a locally free sheaf. 

For the moduli space that corresponds to this one under the diffeomorphism 
between the Xj, however, we must consider sections of Xi of degree —1 with 

respect to the Kahler form of on X C x P'^. There clearly are none. 

(The Casson invariant of Xi that corresponds to lowest degree sections is for 
degree sections, i.e. those of the form P^ x {x} C Xi. Write the equation defining 
the (2,4) divisor Xi = A4i as x^/ + xyg + y^/i = 0, where /, g, h are quartic 
polynomials on P^. Then P^ x {x} lies in S if and only if x lies in the intersection 
of the three quartic surfaces in P'^. Thus we may deform to a case where there are 
precisely 64 isolated sections in the moduli space, and therefore the Casson invariant 
is 64, and again all the sheaves are locally free.) 

Thus we have finally recovered the result that the Xi are not deformation equiva- 
lent, as polarised varieties. Although the result is both weaker than the results using 
Gromov-Witten invariants [ Ru | and essentially a more complicated rerun of the same 
proof (going as it does via the same Gromov-Witten invariants), it is nonetheless 
encouraging that there are examples of the invariant which are calculable yet contain 
highly non-trivial information. 
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